My general opinion that whether a person finds a language easy or not has a lot more to do with whether that language is similar to their first language, than it does with the actual complexity of the language.
The issue is that some languages just are very different than most other languages a person may know. For example I don't think Finnish is significantly more difficult than German (that one amusing meme notwithstanding), but it's not Indo-European at all, so it's going to be further removed for the vast majority of foreign learners than German is.