Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

4 thou isn't a particularly high degree of precision.

For about five bucks you can get feeler gauges more accurate than that, and for about thirty you can get a micrometer that's accurate to 0.0005" over a whole inch. (I checked mine against a set of grade 2 gauge blocks at their calibrated temperature.)

Normal CNC machines, not designed for extreme precision but the sort of thing an exceptionally serious hobbyist or decent makerspace might have just sitting around, will happily hold 0.001" across the entire working envelope, which may be the volume of a couch cushion or larger. The one I'm familiar with (which dominates one corner of the local makerspace) has thermal sensors scattered about the large castings that make up the machine frame, so it can compensate in software for the estimated warpage of the frame depending on how the HVAC has been blowing on it. And that's mid-90s tech, things have only gotten better since then.



There's a "high tech" (i.e. they mostly don't use cutting tools) machining shop nearby; their proprietary machines (designed & built in-house, never sold or leased externally, only used for contract manufacturing) are repeatable within a few µm (they offer tolerancing below 5 µm on finished parts) and move about as quick as the rapid traverse on a more standard CNC milling machine.


Nanotech Systems (Moore's current name) make a high-precision drum lathe with 1nm positioning precision, with a 2600mm working area [1].

1 part in 2,600,000,000.

Dan Gelbart built his own lathe with <1µm final part tolerance [2].

Within a few thousandths of an inch isn't very impressive. I can hold 1 thou on a cheap manual lathe (Grizzly G0602 10"x22" working area), with a decent amount of care. Holding to tenths (1/10000") is impossible with the equipment I can afford though.

[1] https://nanotechsys.com/wp-content/uploads/2019/12/HDL2600-B... [2] https://www.youtube.com/watch?v=sFrVdoOhu1Q


That's resolution, not precision or repeatability, although such high resolution is very impressive. I'm not sure how exactly a glass scale is supposed to work to 34 pm, I assume these use analog interpolation between marks or perhaps some kind of interference mechanism.


EDM?


Laser


The most accurate measuring device at the jet engine MRO I worked at was a CMM. It had an accuracy of 2 micron, 0.00008"

It was a robotic arm that lived in a temperature controlled room. The parts it was to measure were left inside the room to acclimatise. When they were at the correct temperature, you would put a part on the largest granite block in the southern hemisphere that was so flat they used lasers to calibrate it. Then run the program for the part. The arm would move into position and then a little probe would touch the part to get a measurement. It was used to measure things like warpage of inner diameters etc.

Funny trivial, if you ran the wrong program the arm would end up colliding with the part causing the probe to break. There was always spare probes kept because this happened. $20k each.




Consider applying for YC's Summer 2026 batch! Applications are open till May 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: