Pretty fundamental:
1. Binary search. Binary search. Binary search. This is an incredibly important concept, and it pops up in everything from convex optimization to databases. If you need to find a particular value in a big set of things, and you can figure out which direction you need to go, use binary search for a MASSIVE speed up.
2. Matrix operations. Now, this one only pops up for me because I’m more of a data scientist, but in general, it’s important to remember that matrix multiplication is faster than you think it is. Brilliant people have come up with incredibly unintuitive algorithms to make matrices multiply in O(n^2.something) instead of O(n^3), like you would expect. In practice, these algorithms aren’t necessarily used, but the BLAS and MPI libraries do provide incredibly fast matrix multiplication all the same. Anytime you can express a bunch of calculations in terms of matrix operations, you win.
3. Linear regression. Simple, beautiful, and useful.
4. Caching. Justuse caching. Does that count as an algorithm? I don’t know, but it is the cause of, and solution to, all of your problems.
5. The idea of a concurrent work queue.
6. Knowing when to throw hashtables at a problem.