This is pretty cool, absolutely mind-boggling to look at those long reference lists. There are a few more which I know but not mentioned yet and some I feel are missing some interesting details. And anyone who spot errors please correct me so as not to spread wrong information.
Spoiler for ep 2 - man who balances a red wash basin on his head:
The entry in the japanese wiki is named
赤い洗面器の男 (romanized as "akai senmenki no otoko", meaning "red wash basin man"). Apparently, it is a running gag found in Mitani Koki's work, a famous Japanese playwright in Japan. Most of them appeared in Furuhata Ninzaburo, a popular detective drama serial from the 90's, where Tamura Masakazu (some J-dorama fans should be able to recognize him) plays the title character. The gag is such that a character will talk about the red wash basin man and ask the question "why does he balance the red wash basin on his head?", but the punchline is always never given.
Spoiler for ep 3 - Ladybug Samba:
てんとう虫のサンバ (romanized tentoumushi no samba, meaning Ladybug Samba). That is a classic Japanese song from the 70's usually sung in wedding receptions/dinners. The Ladybug Samba "punk version" is a reference to one of the lines spoken by Oosaki Nana in ep 50 of the NANA TV animation series. I haven't read the manga so I'm not sure if it's originally from there.
Here is a youtube clip of a cover.
Spoiler for ep 4 - figurines on Konata's computer:
On top of Konata's computer is a figure of Kooh, a character from the popular Korean game, Sukatto Golf Pangya. It has the same pose as the market released figure though in a different color. See link
here and
here.
Unfortunately, I still can't identify the other figure. I have a large pic (1.35MB) of the figures, but it didn't help much.
Spoiler for ep 4 - Kanisawa Kinu:
Mentioned by aohige already. Just want to add that the fan which Konata was using is a real Tsuyokiss merchandise. See link
here.
I recall more references some of which kj1980 mentioned in the ep threads, MakubeX2 also mentioned a few including the Ponytail references and one of a game poster, someone noticed that Kagami was reading Toki wo Kakeru Shoujo, Kaoru Chujo verified that the smoking middle age man is Ishihara Tetsuya, and Level E has a very good list on his forum. Seriously there are a lot of references!