In Dearly Devoted Dexter, the second novel of Jeff Lindsay's series about the serial-killer turned forensic specialist turned anti-hero, there is a down-from-Washington character named Chutsky who uses a phrase along the lines of "Those who swim around in this pool end up getting flushed." A handful of pages later, Dexter wonders about this line, and thinks about how it sounds like something out of a bad crime novel. Lindsay, not quite satisfied with the single "rimshot" there, brings up the line again another handful of pages in, and this time Dexter confronts Chutsky with it, and Chutsky admits it was a cheesey line used for effect. For those keeping score, Lindsay has now used a single line three times for three different reasons. Two of those reasons, though, deal with "lampshade hanging". This is when an interneral cog in the gears of a fictional device point out that the machinations of the plot are not 100% on board with sanity.
Have you ever been watching a movie and the characters do something stupid? Of course you have. There are almost no movies on the market without at least one scene of stupidity. Ok, then, have you ever seen this stupid movie scene and then, right about the time you are turning to your date to point out how silly it is, one of the characters says, "This is like a stupid scene in a movie!" and you feel the rug ripped under your feet? The movie has just acknowledged it is stupid, now the only attack you have in your arsenal is, maybe, "Yeah, what he said!" (It does bring up the question, though, if they admit is stupid but play along, doesn't that just bring it back around to stupid? Let's not ask such questions, though, because that is the way to madness...) In 28 Days Later, as the survivors are going to leave the city, Frank decides to take the tunnel filled with cars and unknown darkness. Jim, like most every movie-goer currently watching, proclaims that it is a "shit idea". How does he know that? Because it is "quite obviously a shit idea." That is a prime example, and a decent one in-context, of lampshade hanging.
How about this? You are watching a movie and a character has to do something silly, like sneak into an ex-girlfriend's house to steal an antidote to cure a current girlfriend. In the middle of the prolonged sneaking scene (which, of course, ends in failure because this is Hollywood); the best friend, along for the ride, says "This feels like the plot in some dumb romantic comedy..." and the audience gets a laugh. In Jay and Silent Bob Strike Back, there is some joke about "Who would pay to watch two hours of fart jokes?" and then the actors all turn to look at the camera (and therefore, "at" the audience). Again, everyone laughs, except the four or five people who agree and get up and walk out. Lampshade hanging.
By the way, this is not a new thing, for those thinking it is a sign that people are running out of ideas. Shakespeare uses it in Twelfth Night. Act III, Scene 4, Fabian declares "If this were played upon a stage now, I could condemn it as an improbable fiction." Har-de-har-har. I still haven't read Tristam Shandy, but it apparently uses it alot, too.
I personally would divide the old lampshade hanging into two categories: lesser (narrative) and greater (affective). Lesser lampshade hanging would be somewhat realistic characters responding to extreme circumstances. Say the main character stops to help a hitch-hiker, and she gets exposed to some disease. She might say something like, "This does not happen to people like me!" or, "How does being a good person deserve this?" In both cases, she is serving the narrative by pointing out what people might actually say about the situation. The 28 Days Later example satisifies this: Jim is not pointing out how stupid the idea is because he is trying to stop-gap some plot-hole. He is reacting to what's in front of him. In the lesser range of narrative lampshade hanging, you have people pointing out things like: "I thought I checked the seal," as explanation about how the spill occurred. It feels natural, and can help to work out some moments that would be awkward, but nevertheless satisfy "It's In the Script" (IITS).
Greater lampshade hanging, or affective lampshade hanging, is for an effect, likely a comedic one. Why not make it a two-fer on a ridiculous plot and have the audience chuckling along? The Jay and Silent Bob example is about as extreme as you can get, without having, say, Kevin Smith, break out of character and explain it. I call it "greater" not because it is more effective or because it is the better use, by the way. Instead, I consider it a greater-in-width sort of bandage on the plot. Comedic movies, especially those with post-modern characteristics, can get away with it, but it can be kind of jarring with most other styles. It irritates me, half the time, unless it is done just right. Your personal mileage may vary.
I should point out that a sister use of lampshade hanging falls under this greater category, and it shows up more often in genre films (in my experience), and that is where a character points out a huge flaw in the plot not for comedic effect and not necessarily to be more realistic, but simply because having it acknowledged and out in the open can presumably make the audience more at ease with it. A good example of this occurs in The Core when they name the impossible material "unobtanium". Unobtanium is the fan name given to any SF alloy that has almost mystic properties, and whose structure and make-up are glazed off by script writers. By naming the material unobtanium, they try and patch that plot-hole, while simultaneously getting buddy-buddy with the audience. Make note, mind you, that this is a movie that shows regular birds flying through car windows, and so pretty much any attempt at reasonability is just silly.
As for the name, one explanation I have heard is that glaring plot-holes, or even minor ones, can act like a bare bulb on the audience's mind. It breaks them out of the moment. Therefore, you hang a lampshade on the bulb, and this helps to dampen, though likely not block, the glaringly obvious. For me, it's all in the rhythm of how it is used. If I ever do write a movie, by the way, I am going to use it and then someone is going to point out that it sounds like someone is lampshade hanging. I will be the meta-irony king.
To end, I'll leave you for a couple links and one quick return to my opening paragraph. Linsday gets away with both greater hanging (in the "bad fiction" line) and lesser hanging (when Chutsky acknowledges the silliness of the line). I'm not sure if that gets a "well done" or "hiss". Now, the links:
Si Vales, Valeo