The New York City bash was also attended by Alec and Hilaria Baldwin, Amy Poehler, Jon Hamm and wife Anna Osceola, Chris Rock, Bill Murray and more

Holiday magic was in the air for Chris Evans and Alba Baptista on Thursday night.

The newlyweds made their first appearance as a married couple at a holiday party in New York City, hosted by Evans’ Marvel costar Scarlett Johansson and her husband, Saturday Night Live star and writer Colin Jost.

Baptista, 26, wore a red midi dress under a long black coat, with her brunette hair pulled back into a low updo. She finished the ensemble in strappy black pointed toe heels and a white scarf, carrying a black shoulder bag decorated with a red floral pattern and rocking a bold red lip.

Photographed holding hands with his wife, Evans, 42, kept to holiday colors in a dark-green shirt over burgundy slacks, also wearing brown shoes, a dark tan coat and brown-rimmed glasses.

Never miss a story — sign up for PEOPLE’s free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human-interest stories.

Chris Evans and Alba Baptista attend Scarlett Johansson and Colin Jost’s Christmas party at the Flatiron Room on Dec. 21.

Inside Jennifer Lopez and Ben Affleck’s Holiday Bash: Christmas Carols, Outfit Changes and A-List Guests (Exclusive)

The party was a star-studded affair also attended by the likes of several current and former colleagues of Johansson, 39, and Jost, 41, including many SNL alums.

Alec and Hilaria Baldwin, Amy Poehler, Jon Hamm and wife Anna Osceola, Kieran Culkin and wife Jazz Charton, Chris Rock, Bill Murray, Sarah Silverman and more were spotted either arriving to or exiting from the festivities.

Other famous faces in attendance included Lorne Michaels, newlyweds Greta Gerwig and Noah Baumbach, Rachel Dratch and Seth Meyers with wife Alexi Ashe.

Scarlett Johansson and Colin Jost attend a Christmas party they hosted at the Flatiron Room in New York City on Dec. 21, 2023.

As for Evans and Baptista, a source told PEOPLE in September that the two had gotten hitched at a private estate in Cape Cod, Massachusetts.

Many celebrity friends were also spotted in nearby Boston for the ceremony, such as Evans’ Marvel costars Johansson, Robert Downey Jr. and wife Susan Downey, Chris Hemsworth and wife Elsa Pataky and Jeremy Renner, plus John Krasinski and Emily Blunt.

Chris Evans; Alba Baptista.

Chris Evans and Alba Baptista’s Relationship Timeline

Evans, who was confirmed to have been dating Baptista for “over a year” in November 2022, opened up about married life in October, revealing that the couple also had a ceremony in Portugal.

“It’s a lot planning a wedding. For those of you who are married you know it takes a lot out of you,” the Ghosted actor said at New York Comic Con 2023.

Added Evans, “But now that we’re through that, we’ve just been enjoying life, gearing up for autumn, my favorite season. Now we’re just relaxing and enjoying life and reflecting.”

A source confirmed to PEOPLE in September that things between Evans and Baptista “got serious very quickly” after they first met in Europe: “He kept saying he knew she was the one.”

“He has been ready to settle down for a while,” the insider added. “He can’t wait to have a family. He was just waiting for the right girl.”