We are happy that an African team has beaten a good European team and we hope that the whole of Africa is celebrating the continent's first win in the 2010 World Cup," Rajevac told a post-match news conference.
"It was a very difficult match and a clash of contrasting styles, so I have to congratulate my players for a valiant effort because they kept going until the end," he added.
But Rajevac also had mixed emotions about beating Serbia in their opening Group D match and said he hoped both teams would qualify for the last 16.
"This is the greatest win of my career but I also feel sorry for Serbia, I saw the anguish on the players' faces after the final whistle.
"I had a job to do and I did it professionally but I hope my countrymen win their next two games against Germany and Australia, I wish them the best of luck."