Mike Stoller on the Musical Universe of the Brill Building

Mike Stoller, left, and Jerry Leiber in 1959. Photo: David Attie

As told to Rebecca Milzoff

I was born in New York City, and I lived here until I was 16. One of my earliest memories having to do with music in New York was hanging out on 52nd Street at the jazz clubs in the 1940s, like the Three Deuces; the Famous Door; the Onyx Club; Jimmy Ryan’s, where I would go to hear Charlie Parker and Miles Davis, Lucky Thompson, Erroll Garner. I would go up to the bar and in my deepest voice—with my schoolbooks under my arm—say, “I’ll have a beer.” I think if you could reach the bar, they’d serve you a beer. It was thrilling.

Jerry and I started writing when we were 17 in L.A., and when we arrived at the Brill Building, we’d had a number of hits already—“Hound Dog,” and “Kansas City,” “Searchin’,” and “Young Blood,” which was a million-seller single. We moved to New York in ’57. We didn’t have an office at first; we worked up at Atlantic at 157 W. 57th. At a certain point in 1960 we moved to an office at 40 W. 57th, which is now an enormous building. At that time it was a five-story, little skinny building with a corset shop on the ground floor, and we were on the fifth floor. We had just the two of us and some oak furniture we bought at some surplus place with an upright piano. It had a skylight, so it was very noisy when it rained. But then we decided, we’ll make the move, and in 1961, I think, we moved to the Brill Building on the ninth floor. Brill was 1619 [Broadway], at that time between two eating establishments: the Turf, which was right on the corner of 49th and Broadway, which had walls covered with sheet music from the early ’40s and ’50s; on the other side of the entrance was Jack Dempsey’s restaurant—the better-heeled and gamblers would have lunches there. The Turf was a hangout for a lot of musicians. If someone needed a bass player to do a demo, they could call or run over to the Turf and see who was there. “Hey, bass player—I need you!” From there we did not only our writing but our rehearsals and record production. Of course the building was amazing in those days; almost every floor was inhabited by music publishers, songwriters, band leaders, talent agents—it was quite an interesting place. And then there were guys on the street who used the phone booth as an office. Seriously, they’d give out the phone number, and people would stand outside waiting for a phone call, trying to scare other people away from coming in to use the phone. Right next door to us, at first, on the ninth floor, facing downtown, was the office of Irving Caesar. Bobby Darin moved in when he moved out. Irving was in his late 60s; he wrote “Swanee” with George Gershwin, and “Tea for Two.” We’d see him in the elevator. He’d be in early in the morning and then we’d see him on his way to head for the track. It would seem that many people in the music business were gamblers. Johnny Marcks was in the building, who wrote “Rudolph the Red-Nosed Reindeer.” Certainly he did very well—I guess every December he made enough to pay for a year’s rent, at least. People would come up to the 11th floor and work their way down the staircase knocking on every door, trying to sell a song, and in some cases there were some pretty good ones and occasionally someone with a catchy tune would sell it on one floor for an advance and then get another advance on another floor, and have a really good weekend. The restrooms on the landings were used a lot by vocal groups because the tile walls made a good echo. A lot of R&B groups would rehearse there before knocking on a door. There were a lot of very good writers circulating who either had a publisher in the building or came from 1650 Broadway, which is where Don Kirshner’s office was, writers like Carole King and Gerry Goffin and Barry Mann and Cynthia Weil. We were producing, publishing, writing. It was 18 hours a day, and all the real action was in the Brill Building. Then it carried over to Al and Dick’s Steak House after six o’ clock, at 54th just east of 7th Avenue. All the music people hung out at the bar, mostly putting each other down. The A&R men from the big companies, the producers, the promotion people, the songwriters—it was a madhouse.

We knew a lot of writers. Doc Pomus, who we’d met before; his partner, Mort Shuman. Otis Blackwell, a wonderful writer who wrote “Don’t Be Cruel.” And of course Burt Bacharach. We signed a fella named Jeff Barry. A gal came in and was playing songs, and we told her she could use the office as long as we had a first crack at her songs, and that was Ellie Greenwich. Barry and Ellie started writing together, in our office, and we signed both of them. Ellie had a knack for writing simple but effective tunes. She and Jeff wrote really catchy tunes, and they became very adept at production as well. And of course Phil Spector, who had been a protégé of Jerry’s and mine. He was arrogant, and also talented—a talented guitar player. He was not too pleasant. We got him his first production job, which was offered to us when we were very busy, and that was to do Ray Peterson. He did “Corinna, Corinna,” which became a hit. And then not long after he signed with Atlantic, and they started to give him some of the artists we were working with. He wrote a song with Jerry, a very good song, “Spanish Harlem.” As a producer, I’m very proud of that record. On that session, we used Phil as a fifth guitar player and he has since told everybody he produced the session. He put his arms around everything and claimed they were his. He went on to take credit for everything Jerry and I did, including records we produced before he ever got to New York.

Listen to our Spotify playlist of hits from the Brill Building

*This is an extended version of an article that appeared in the March 24, 2014 issue of New York Magazine.