The real antipathy, I think, started with the Crusades - a common pastime or recreational activity for Crusaders was to conduct a pogrom - find a shtetl (Jewish village,) rob whatever you could find useful, kill the men, rape the women, etc. It was OK for Crusaders to do so, because they were attacking Jews, not people.
I think the root of/excuse for that sort of idea was the idea of "usury." It's also the root of the modern prejudice of Jews controlling the world's wealth (and by extension, media.) The proscription against "usurous interest" meant that Christians couldn't lend money for profit - their interpretation was that any interest was usurous interest. Jews didn't have such a proscription; even though medieval Jews were bound by religious reasons to limit their profit from the transaction to something that was considered fair all around, the fact that they made anything at all made them - in some Christian eyes - greedy bloodsuckers of Shakespeare's Shylock mold. It also made them a sought-after part of life in those days, but that bit sort of got swept under the rug.
Then, of course, once Sprenger and Kramer became popular, the Dominicans sprang up and began to conduct the Inquisition. While the Malleus Maleficarum almost exclusively targeted women, the Inquisition spread it's umbra to include the arbitrary term of "heresies" - which, of course, included being anything other than Catholic.