Yes, yes, yes! You have asked the question at the right place. Muppet Stuff was an excellent store that folded in the mid-late 80s. The official year was mentioned in a thread last year.
I first learned of Muppet Stuff from an ad in Muppet Magazine. My family took a trip to Manhattan in when I was 7 and we visited the store. They'd always have a cool window display (as you might expect) and little Kermits were hand painted on the inside wall trim. The store was tiny.
The inside architecture wasn't remarkable, but they had Muppets stuffed in every square inch! If you'd like to see what it kind of looked like, flip to the collectibles page in The Works book. It kind of looked like that everywhere. Everything from Fraggle figures and wind-up Doozers to Muppet soaps, pencils and stationery. It was not a Sesame store, but there was some Sesame stuff. The Muppets, Muppet Babies and the Fraggles were the stars of this shop.
My Dad and I used to take a trip to NY once a year to see the shows and drop by Muppet Stuff. One year we went by and it had closed. I was about 14. I don't know why they couldn't keep such a small store open - especially with its mail-order business and NY location. It's now a flower shop.