Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

internally it will probably be approved, for example I am sure that Google drive applies basic compression and deduplication to uploaded files, but if I upload 10 files of 10GB of zeros they are gonna count 100GB, not the few MB they are actually writing to disk

(there are good reasons for this, but still declared consumption is different from internal consumption)



Log Analytics uses a columnar compression format on-disk, so ingested data is likely compressed by anywhere between 10:1 and 100:1, maybe even higher.

However, the wire format is super verbose JSON.

They bill per GB of the latter, not the former.

To put things in perspective: How many $ of CPU time do you imagine it takes to column-compress 1 TB of data? I would estimate that a single modern CPU core could do this in a minute or so. Factor in various inefficiencies and make it a super generous 1 hour. At spot pricing, that's about $0.01! One cent!!!

The larger cost would be bandwidth. Azure charges a huge markup for traffic (just like AWS), so for example zone-to-zone data costs $10 per terabyte at retail pricing (not internal costing).

They store that data for 30 days "for free" (lol). Assume a worst-case compression ratio of 10:1 and then that means that they have to retain 100 GB for 30 days. That's $9.43 for a Premium SSD at retail pricing.

So their hosting costs for Log Analytics is something like $20 per TB ingested, but they charge well over $2000 for it.

That 100:1 markup is pretty sweet if your KPIs are based on recurring revenue.

There is no way in hell they will ever "optimise" this. Any accidental improvement will be rolled back or "adjusted" to ensure the revenue stream doesn't fall off a cliff.

Have you not wondered why it's taken them so long -- over ten years -- to enable any feature to filter logs at the source?




Consider applying for YC's Summer 2026 batch! Applications are open till May 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: