Units, precision and tolerance

Every number in a model has a unit, and every number in a computer has limited precision. Both cause real bugs: a wall that is 304.8 times too long, two points that “should” be equal but are not, a model that jitters in a viewer. This lesson covers the three habits that prevent them.

Internal units and display units

BIM software stores lengths in one internal unit and converts only when it shows or reads a value. Revit, for example, stores every length in decimal feet, even in a metric project; the millimetres you type are converted on the way in. IFC files declare their units in the file, and a reader must apply them.

1const double MmPerFoot = 304.8;2 3static double MmToFeet(double mm) => mm / MmPerFoot;4static double FeetToMm(double ft) => ft * MmPerFoot;5 6double wallFt = MmToFeet(3000);7Console.WriteLine(wallFt);8Console.WriteLine(FeetToMm(wallFt));

9.84251968503937
3000

Floating point is not exact

A double stores about 15–17 significant digits in binary. Many simple decimals, such as 0.1, have no exact binary form, just as 1/3 has no exact decimal form. Each operation rounds to the nearest representable value, and those tiny errors add up.

1double sum = 0;2for (int i = 0; i < 10; i++) sum += 0.1;3 4Console.WriteLine(sum);5Console.WriteLine(sum == 1.0);6Console.WriteLine(0.1 + 0.2);

0.9999999999999999
False
0.30000000000000004

Geometry hits this constantly. Rotate a point a quarter turn and the X that should be zero is not quite zero:

1double angle = Math.PI / 2;                     // 90 degrees2double x = 10 * Math.Cos(angle), y = 10 * Math.Sin(angle);3 4Console.WriteLine($"({x}, {y})");5Console.WriteLine(x == 0);

(6.123233995736766E-16, 10)
False

Compare with a tolerance

The fix is to never test computed coordinates with ==. Ask instead whether they are close enough: within a tolerance chosen for the job.

1using P3 = (double X, double Y, double Z);2 3const double Tol = 1e-9;4 5static bool Near(double a, double b) => Math.Abs(a - b) <= Tol;6static bool SamePoint(P3 p, P3 q) => Near(p.X, q.X) && Near(p.Y, q.Y) && Near(p.Z, q.Z);7 8double x = 10 * Math.Cos(Math.PI / 2);9P3 wallEnd = (x, 10, 0);10P3 nextWallStart = (0, 10, 0);11 12Console.WriteLine(wallEnd == nextWallStart);13Console.WriteLine(SamePoint(wallEnd, nextWallStart));

False
True

Picking the tolerance is a modelling decision, not a maths one:

• Too small (say 1e-15) and normal rounding errors make equal points look different, so walls fail to join.
• Too large (say 1 mm in a model measured in mm) and genuinely different points merge, so small details vanish.
• A good absolute tolerance sits far above rounding noise and far below the smallest real detail. For building-scale models in metres, 1e-9 to 1e-6 is typical; BIM kernels use a fixed tolerance tied to their internal unit.
• For values that span huge ranges (areas of a bolt and of a site), a relative tolerance, Math.Abs(a - b) <= tol * Math.Max(Math.Abs(a), Math.Abs(b)), works better.

float vs double: precision depends on distance from the origin

Floating-point numbers are dense near zero and sparse far away. The gap between neighbouring representable values grows with the size of the number. A float has only about 7 significant digits, so 10 km from the origin, measured in millimetres, it cannot even represent a whole-millimetre step reliably.

1float farF = 10_000_000f;          // 10 km in millimetres, as float2double farD = 10_000_000.0;        // the same, as double3 4Console.WriteLine(farF + 0.3f == farF);5Console.WriteLine(farD + 0.3 == farD);

True
False

Adding 0.3 mm to the float changes nothing at all: the detail is lost. Try it below: move the slider from the origin out to site scale and watch the gap between representable values.

Round for display, not for storage

Show 6.12E-16 as 0.00 with :F2, but keep the stored value as it is. Rounding stored coordinates every time adds a new error on each edit and can push points across a tolerance boundary.

1static string ToFeetInches(double mm)2{3    int totalInches = (int)Math.Round(mm / 25.4);4    int feet = totalInches / 12;5    int inches = totalInches % 12;6    return $"{feet}' {inches}\"";7}8 9Console.WriteLine(ToFeetInches(3000));10Console.WriteLine(ToFeetInches(2743));

9' 10"
9' 0"