There was a thread about Sesame Street Live in the Sesame Street section where somebody scanned every picture from the program book for Big Bird's Totally Amature Show. For that one, it seems like it was just atalent show put on by the Sesame Street muppets.
I don't know about the first one, but it seems like it must have been a mystery of some sorts. I've seen a listing of characters from it, and for some reason, for a first Sesame Street Live show, it didn't have very many Muppets from the show. It seems like it had a lot more new anything muppet-type characters and some of the main Sesame Street Muppets. I know that it had Big Bird, Ernie, Bert, Grover, and Cookie Monster, and I think it had The Count and Oscar, and that's basically it. Since it was a mystery, I thik that Sherlock Hemlock should have been involved.
The second one did have a lot more characters, including some who I think rarely appeared in Sesame Street Live. It is one of the few to feature Herry Monster, as well as Gladys, Fred the Wonder Horse, the Two-Headed Monster (I wonder how a walk-around of him was performed), Simon Soundman, and many others. Interestingly, Louise Gold performed voices in this production even though she wouldn't perform on Sesame Street untill 1991.