Tentative at times, Ivanovic seemed to regain her edge midway through third set. She led 40-0 in the fifth game and was about to break Coin's serve when suddenly the momentum shifted.
Coin came back to hold, starting her decisive streak.
Ivanovic tried to stave off Coin in the final game, but it was too late for the 20-year-old Serbian star. Coin won on her third match point, quite a result for someone playing in her first tour-level event.
Coin had failed to qualify for the Australian Open, French Open and Wimbledon. And she certainly wasn't anything bankable going into this match. She'd earned less than $100,000 (euro67,700) lifetime as a pro, and was facing someone who'd won nearly $6 million (euro4 million).
Next up for Coin in the third round is countrywoman Amelie Mauresmo, who beat Kaia Kanepi of Estonia 2-6, 6-4, 6-0. Coin and Mauresmo once played at the same club near Paris and shared the same coach, though they don't know each other well.
"I know she was No. 1," Coin said.
Going into this Open, Ivanovic had played only two matches since Wimbledon while her thumb healed. The injury forced her to withdraw from the Olympics before they began, limited her practice time, and sent her from Beijing to Australia for treatment.
Fourth-seeded Serena Williams routed Elena Vesnina of Russia 6-1, 6-1 and seventh-seeded Venus Williams overwhelmed Rossana de Los Rios of Paraguay 6-0, 6-3.
"I'm very satisfied so far, the way it's gone," Venus said.
Venus breezed past an opponent ranked 117th. After teaming with her sister to win Olympic gold in doubles, she stayed on course to play Serena in the quarterfinals here.
"Whichever way the draw goes, whichever way the matches go, as long as hopefully it's a win for me, I'm pretty happy about it," she said.
Williams is the last woman to win consecutive championships at the US Open, but hasn't taken the title since 2001.
"Oh yes, I remember. I won't forget, but I'd like to have a more recent memory as of, like, '08," she said. "Kind of overdue."
Among the other winners were No. 9 Agnieszka Radwanska, No. 17 Alize Cornet, No. 18 Dominika Cibulkova and No. 19 Nadia Petrova. But No. 20 Nicole Vaidisova lost to Severine Bremond of France 7-5, 6-3.
The No. 6-seeded Safina, not nearly as volatile as brother Marat Safin, played under control in beating Roberta Vinci of Italy 6-4, 6-3.
On the men's side, fourth-seeded Ferrer beat Andreas Beck of Germany 4-6, 7-5, 6-3, 7-6 (5), sixth-seeded Murray defeated Michael Llodra of France 6-4, 1-6, 7-5, 7-6 (7) and Argentine teenager Juan Martin del Potro notched his 21st straight win, beating Thomaz Bellucci of Brazil 4-6, 6-1, 7-5, 6-3.
