The bugger for an English speaker is that English is pretty much the easiest language in the world, so *any* other language is going to be harder.
If you can cope with the pronunciation, I think grammatically French is fairly straightforward.
German (particularly if your own dialect is north of England) is very easy to pronounce and shares the same vowel sounds and stress patterns, but the grammar is (moderately) difficult. The word order can feel quite alien to an English speaker (see Gnome's video upthread), and it's inflected (the word "the", for instance, changes not just according to grammatical gender, but according to the function the noun has in the sentence - subject, object, indirect object etc.)
English has a bit of this left over in pronouns:
"I gave his book to him."
He gave my book to me."
But in German this extends to
"The ( *nominative form of the* ) man gave the ( (accusative form of the* ) book to the ( *dative form of the* ) other man."