It's perhaps worth noting that HTTP has essentially the same unspecified directory list problem as FTP. I've seen programs that screen scrape apache etc directory lists (as displayed when there is no index.html). Oddly, gopher was an intermediate protocol that got that right.
HTTP wasn't really intended as a file/directory transfer protocol the same though. The problems with FTP means that is fails at its primary function (unless you apply copious work-arounds).
Many moons ago in my younger days I wrote an FTP based file/tree synchronisation tool. I have since vowed to never touch FTP again.
