For me, stealth is a very mixed bag. I really appreciate games that allow you to use stealth, but don't completely shove it down your throat with unnecessary and unforgiving trial and error sequences. The early Splinter Cell games were a perfect example of this in action. It's very hard to get stealth "right," especially because you need the enemy AI to act appropriately to what we would expect in real life, but I think I've found three games that seem to match this well. Don't forget that I attach videos to the pics!
Metal Gear Solid 2
I would normally say the Metal Gear series overall, but I wanted to highlight this one for a few reasons. First, let's talk about the MG games up to this point. You have the original NES version (the MSX version is better), where you just have to watch line of sight and not use noisy shit, evading enemies is fairly easy, but it's also easy to die. You have Metal Gear 2 (MSX), where there are noisy surfaces, you can crawl to mitigate noise, you can hide under things, and overall stealth is a much more desired trait.
And of course, you have the original Metal Gear Solid where there's an incorporation of all these elements along with pressing against surfaces, tapping on walls, the cardboard box (which was also used in the last one), more noisy surfaces, footprints in the snow, and all kinds of clever shit. In my opinion, MGS2 is the apex of Hideo's stealth action genre. Now, I say this not having played MGS4, so keep in mind some of what I discuss may in fact be better in the 4th game.
Everything that occurs in the previous MG games is taken into account, plus there's a new hoisting/hanging system. You also have the ability to make people surrender at gunpoint, but different guards will react differently. You can drag around bodies and hide them. Furthermore, you can hide yourself in more locations, but this comes at a price: smarter AI. For the most part, the AI does what it's done all along: moves in on your location and prepares for a shootout.
However, let's say you've fled to a locker room and the alarm has gone into caution mode. They will start checking lockers. They will start checking in usual hiding spots. See, in MGS, they didn't do this. As long as you hid somewhere they didn't see you enter, you were safe. In this one, they're more self aware and as a result, it becomes harder to hide. The reason this is especially good is because stealth isn't just about hiding; it's about never being found in the first place.
They won't necessarily find you every time and it's not as trial and error, again, as earlier Splinter Cell games, but it still makes for some damn fine stealth and more than a few hair raising moments when you're wondering exactly where they're going to check.
Deus Ex - Human Revolution
The original Deus Ex was a game that prided itself on stealth. That makes sense. The same team behind Deus Ex also made the Thief series, which is a game that's so heavy on stealth that if you're even a little unaware, you're completely and thoroughly fucked. Of course, Deus Ex was also a game that featured shootouts, continuous improvement of the character in a variety of ways, and so on, so really you didn't even NEED to use stealth if you didn't want to, but it was preferred.
I was sure that Human Revolution was going to get this wrong, yet they managed to blow me away. First, the stealth is very traditional. Not a whole lot is done that's new other than cranking the enemy AI. Suffice to say, firefights are almost never a good idea. It's not that you can't win them...it's that the enemy AI is that good. Regardless of whether you're fighting street punks or a small army, you're in a world of fuck if you get in a firefight.
Nevermind that, guard patrols actually make sense, they don't always face just the one direction, and you have a lot of paranoid guards, too. They're smart enough to check everywhere and while you can flank them, they watch for that, too. Overall, the system in DX:HR is a very smart system built on a very classic formula. I'd say it's easily the best traditional stealth engine on the market.
Splinter Cell - Conviction
...which brings us to the best modern stealth engine. Conviction really surprised me. See, I really enjoyed Double Agent before this, but it was set up very much like the previous Splinter Cell games, just with less trial and error, which is really all the formula needed. Conviction, however, blows you away with all kinds of new and interesting mechanics that don't just work, but manage to keep the game hardcore while adding a layer of casualness to it that only adds to the gameplay.
How is this possible? Firstly, we do away with the whole "stealth gauge" thingy. If you're running, you can be detected because of sound. Keeping your head low keeps you in better stealth, but if you're just moving at a normal pace they also can't hear you. Simple. You know they can't see you if you're hidden in a dark area because everything goes black and white except the enemies, which pop out vibrantly.
This is an interesting effect because it immediately lets you know without cross-referencing a bunch of gauges and stuff that they can't see you. Furthermore, it allows you to hone in on them with that much more ease. Another nice touch is if you're detected, then run into hiding, your last known location will pop up as a frozen silhouette, which you can use to your advantage for flanking purposes.
A lot of really clever stuff like that was done in the game which added to the already high production values and made the game more playable without seeming too easy or too punishing. All in all, Conviction is easily the best modern stealth engine on the market and I highly recommend it.
Of course, I also highly recommend all three of the titles today. Sure, I get that MGS2 is a bit dated, but it's still worth playing through if you never did. Stealth has become a major factor of today's gaming, so it's worth looking into as a new genre. Now if only Tenchu could make a major comeback...