Ethically, a code "idea" such as a small pattern or even a custom UI control is typically OK to retain for future use elsewhere. Many of us keep "CommonLib" source libraries containing these kinds of things in a readily re-usable way. Even if not, you remember how you did it, and if it's a good way to do it then do it in other situations where it applies. It's ridiculous to try to assert that you shouldn't be a good coder on one project for one client/employer (including yourself) because you were a good coder on another project and shouldn't plagiarize the ideas you had while working for the other guy. It's called job experience.
Legally, it depends on what you signed when you were hired/contracted. Some companies force workers to sign lifetime NDAs with wording that could be construed to mean "you can't write a line of code for anyone else based on any knowledge of our own codebase". A good lawyer can get the lawsuit thrown out even if you did sign that (seriously, are you expected to hit yourself over the head with a baseball bat until you can't remember working there? Much as I might like to do so for some of my prior jobs it's a completely unreasonable expectation), but it'll still be a pain.
Big no-nos both legally and ethically include copying binaries or source that you specifically developed for one employer and pasting them into a solution for another. While patterns and algorithms are well-known, as are best-practice architectures, coding conventions, etc that can make bits and pieces of two competitors' codebases look similar, a source file that was copied verbatim is plagiarism, plain and simple, and an unmitigated violation of implied or explicit copyright as well as any NDA any competent corporate lawyer would draft. You would have to have stipulated, in anything you signed that would restrict this, at every employer you ever worked for, that you are entitled to retain a "portfolio" of general-purpose code that remains your property independent of your employment. You'd have to be damn good for an employer to still want you if you insisted on that, and the line would have to be very clearly drawn as to what you could and could not claim as your own.
Another fuzzy area where you might be OK, maybe not, is UI "look and feel". The bottom line is similar; while the basic GUI controls like buttons, textboxes, checkboxes, radio buttons, etc were all designed for minimum "surprise factor" and are in the public domain as UI concepts, and there are well-known combinations of same that similarly reduce the learning curve through familiarity, it's probably a bad idea to take a novel UI concept that one company developed (under your watch) for doing a particular task, and bringing it verbatim over to your next job where a new project has a similar need. The Telerik and DevExpress controls, and other third-party non-Microsoft proprietary UI libraries, are sold for money, and so are copywritten heavily; creating a carbon copy, even if you typed every character of the source file yourself, can get you in trouble legally; you have to prove that every single character was written without any more than a cursory (user-level) knowledge of how the third-party control works.