My key theme is horror-- not necessarily gore, though I think it has its place in setting up an atmosphere-- and suspense. It's sort of like hororr is a subset of suspense, because you can't have a good horror film without the suspense, but you can have suspense without horror. Hitchcock is a prime example of mastery of suspense without horror.
So what defines the subgenre as horror?
Monsters are a pretty good bet, but then we have to get into the definition of monster. I tend to look at its primary characteristics-- for instance, Hannibal Lecter (Silence of the Lambs, Red Dragon, etc.) being a "monster" is significantly different than Jason Voorhees (Friday the 13th) or Michael Myers (Halloween) being a monster. F13 and Halloween are pretty solidly horror movies, but Silence and Red Dragon aren't.
To me, it's because Hannibal Lecter is a highly humanized character, where Jason and Michael are very dehumanized. Making the monster human destroys the remoteness of the character.
So key point #1: The monster must be inhuman.
That's not to say it shouldn't look human. In fact, human-looking but not human-behaving characters are positively creepy. Species is a good example. Not the best horror film in the galaxy, but the creatures are creepy. John Carpenter's version of The Thing is probably a better example, and a right scary movie.
Key point #2: The monster needs to be a character.
Seems obvious, right? But consider movies like The Fog and The Mist (both Stephen King stories), where the character is a non-living entity? The Fog is more of a classic horror story where The Mist is more a psychosocial thriller, but either one works to illustrate the point. In The Fog, the fog itself is a character.
And what of the zombies in Romero's Night of the Living Dead? Really, in this movie, they aren't the monsters. The people are the monsters, and because of that I don't really consider it a horror film.
There are movies that I don't think work well. Wolf Creek came out of Australia, and the monster is entirely human. You meet him early in the movie and get to know (and dislike) him early. It's sort of in the genre, but really it's more of a slasher film (i.e. "gore porn") than a horror film. The SAW series and Hostel are in there too, along with Texas Chainsaw Massacre and The Hills Have Eyes.
Then there's Deliverance. Man, I'm really on the fence about this one. I think the way the bad guys are portrayed, one could say they're sufficiently dehumanized to count as monsters, but it's a thin line.
Key point #3: The monster needs to be threatening.
A monster that doesn't actually do anything is merely an annoyance.
Key point #4: The protagonist(s) must be isolated.
This is common to every horror film that I've seen, though it's not just limited to horror-- I think it's a pretty prime element of suspense as well.
It's just not scary when you're in a group of people who will help you, or you can call a friend on your cell phone, but if your cell doesn't work when you're stranded in the middle of the woods, or in the middle of that group of people nobody seems to see you, it becomes a lot more frightening.
Isolation can come in a lot of flavors, and that's why I think that Descent into Madness theme is intriguing to me, because with insanity comes isolation in a slow, delicious way. Plus, it offers a degree of weirdness that is very attractive to the illusionist in me-- Wes Craven's They is a lovely example.
So what kind of things count for you in a good horror movie?