• Longness. This is perhaps the only problem on this list that is rare among natural ("national") languages. Esperanto is long-winded, needing two syllables for some of the most common words.  It really bugs me! Many common ideas like bad (malbona), short (mallonga), fast (rapida), nice (afabla), need (bezoni), and this (tiu cxi) are three syllables, while others like mean (i.e. unkind, malafabla), like (i.e. similar to, simile al), hard (i.e. difficult, malfacila), and slow (malrapida) are four syllables. Many other common words are at least two syllables, including "is" (estas), "as" (kiel/kiom), "so" (tiel/tiom) and "that" (tio/tiu). Numbers are somewhat long, although never longer than English long number forms (which are too long also).
• Its phonetic alphabet includes the difficult-to-type letters ĉ ĝ ĥ ĵ ŝ and ŭ. (there are two substitute conventions: the x-convention, cx, gx, hx, jx, sx and ux., and the h-convention, ch, gh, hh, jh, sh, and u. So, instead of ĉu we would write cxu or chu.)
• In contrast to later constructed languages, some combinations of sounds in Esperanto are difficult or cumbersome for many people.  (combos you'll see include -aktse-, shto-, -ertsa-, -iuy, stsii, ...) You should be able to handle all the combinations--I could--but it takes practise, and some non-English folk have particular trouble with some combinations.
• Its accusative ending, which allows the subject, verb and object to be in any order because the subject, verb and object have different endings.  Though effective, the accusative is hard to get used to, especially since one must apply it to the adjectives as well as the noun. Because SVO (subject verb object) is the most common word order among world languages, other constructed languages typically allow you to use the same form for the subject and object when you use SVO order.
• Its plural adjectives: adjectives must "agree" with the nouns they modify.
• Esperanto has a substantial number of adverbs that do not follow the grammatical ending conventions.  It can be difficult for a beginner to tell the difference between prepositions, conjunctions and irregular adverbs.
• Esperanto has a "correlative table" which, while not too difficult to learn IMO, generally doesn't use the grammatical endings.
• Individually named months and weekdays. They should have been numbered or otherwise named so as to be easy to remember. The month names all resemble those in English but the weekdays do not. Further, they are not marked in any particular way, so if you see a noun for an unknown weekday, you won't realize you're looking at a weekday. Even English beats Esperanto here, since the weekdays are all immediately identifiable as days.
• Single-word idioms: although there are few multi-word idioms, there are many compound words whose meaning is slightly or very different from what its component morphemes suggest.  For example, al-igx-il-o, apparently meaning "a tool for becoming to" (when I saw it, I thought perhaps it meant "a tool for transportation") means "application form". A more typical example is "kompreneble", which literally means "understandably", but as best I can tell, actually means "of course". Such nuances mean that you can easily think you understood something perfectly, when in fact you've just encountered an idiom and did not realise it.
  
  Sometimes, these idioms are a necessity, because the language lacks any other means of expression. For example, Esperanto has no word for "should", so instead people use the conditional form of "must", namely "devus", which literally means "would have to", and, indeed, can mean that in some cases.
• Transitivity is unpredictable.  Some verbs are transitive and others intransitive, with an affix required to transform one to the other.  For example "movi" ("move") is transitive so you should add the igx suffix in the intransitive sentence "I move"; on the other hand, "flugi" ("fly") is intransitive, so you should use the ig suffix to make it transitive in "I fly the plane". On the other hand, if you use the transitive suffix on a verb that is already transitive or the intransitive suffix on a verb that is already intransitive, you get different results. "movigi" means "to cause to be moved"; "havi" means "to have", so havigi means "to cause to be had", e.g. "mono havigas felicxecon" = "money causes hapiness to be had".  "flugigxas" means "to become flying", slightly different from "flugi", "to fly".
• Unpredictable relationship between different parts of speech.  In Esperanto, you change the part of speech by changing the ending.  So if I have the adjective blua (blue) and change it to the noun bluo, then what does the noun mean? Does it mean 'something that is blue'?  'the color blue' itself? Who knows!  There are no hard-and-fast rules.  Mangxi means "to eat", while the noun mangxo means "eating" (not e.g. food); on the other hand, rajto means "a right" (i.e. a freedom) while rajti means "to have a right" or "to have the right".  rakonti means "to tell" (typically, to tell a story, but not necessarily), while rakonto means "a story". Strecxi means "to stretch" while strecxo means "stress" (including mental stress).  The meanings of different parts of speech are always related, but not very predictable.

For English speakers, luckily, the meaning very often matches English.  For example, crono means "a crown" and croni means "to crown", which means the same in English as in Esperanto: to corronate.

An extra problem comes when forming adjectives, compound words, or adding affixes; it is not always clear immediately whether a morpheme is intended to have its noun meaning or its verb meaning. Does "konata" mean "known" (participle of the verb koni) or "an acquaintance" (from the noun konato, a person who is known)?

Because of this confusion, it seems to me that Esperanto speakers tend to be careless in their word formation, making Esperanto substantially more difficult to read than to write.  It's not so much that you can't figure out what something means, just that you have to think about it for a minute, and interpret words according to context.

• Vagueness: Esperanto tends to be imprecise in various ways. For example, one cannot differentiate different kinds of adjectives. For example, "ethics debate" and "ethical debate" both translate to "etika debato". Another issue is that some words have too many meanings.  For example, de means from, of OR by.  Thus "gxi estis sxtelita de mi" could mean "it was stolen from me" or "it was stolen by me" (however, the latter is unlikely, simply because passive voice is rare in Esperanto when expressing an idea that has both a subject and an object). Lasi [apparently] means either "to leave" or "to allow", depending on context.  There are many other examples.
• Thanks to careless speakers importing words from their languages, Esperanto has a substantially larger vocabularly than it needs: there are now over 9,000 roots considered "official", and many more are actually in use, although a less than 2,000 is needed for most speech.
• There are various other minor issues, often concerning individual words in the language. For example, the term "English" as in "English is wacky" cannot be a noun in Esperanto! Only the adjective "angla" refers to the English language; the corresponding noun "anglo" refers only to English people.
• Lastly, no one seems to know of a good English-Esperanto dictionary! At least not one accessible on the internet. It's not Esperanto's fault, really, but it can't be good for the movement.