Tolerancing Standards and Metrology Methods Used to Verify Custom Optical Lens Quality

When you order custom optical lenses, the drawing looks perfect. The coating specs are right. The substrate material matches your application. Then the parts arrive, and your system won't focus. The problem is rarely the design. It's the tolerancing standards and metrology methods used to verify custom optical lens quality that determine whether a lens actually performs as specified. Relevant specifications and application guidance are available through Laser Optical System.

Most buyers assume that a lens manufacturer measures everything. In reality, many shops only check a handful of parameters, and they use different standards than you expect. A lens that passes their inspection may fail yours. This tutorial walks through the tolerancing standards that govern optical component manufacturing, the metrology methods used to verify custom optical lens quality, and how to audit your supplier's inspection process so you get parts that work the first time.

Key Takeaways

• ISO 10110 is the global standard for optical drawing tolerances; ANSI/OEOSC and MIL-PRF also appear in US defense and commercial work.
• Surface form errors (irregularity) and surface roughness affect performance differently — know which one matters for your application.
• Interferometry measures surface figure to fractions of a wavelength, while profilometry handles roughness and edge geometry.
• Centration error, not just surface quality, determines how well a lens performs in imaging and laser systems.
• A proper inspection report includes measured values, test methods, and environmental conditions — not just a pass/fail stamp.

What You Need Before Starting

Before you can verify custom optical lens quality, gather your design documentation and understand what your supplier should be measuring. You need:

• Your optical drawing with tolerance callouts (ISO 10110 format or legacy ANSI format)
• The lens specification sheet from your supplier, including coating performance data
• Knowledge of your application's critical parameters — focal length, wavefront error, or transmitted wavefront distortion
• A basic understanding of the test equipment used in optical metrology: interferometers, profilometers, and autocollimators

If you're sourcing components for a demanding system, review the product range available from experienced manufacturers. Our main products include optical window,prism, lens, beamsp littler, filter, wedge, blanks and etc., and most custom optics suppliers will provide inspection data upon request. Ask for it before you place the order, not after.

Step 1 — Understand the Tolerancing Standards That Govern Optical Lenses

What to Do

Familiarize yourself with the three main standards families used in optical manufacturing:

• ISO 10110 — the international standard for optical drawing tolerances. It uses a block system where each tolerance type gets a number: 3/ for surface form, 5/ for surface texture, and 4/ for centering errors.
• ANSI/OEOSC OP1.002 — the American standard for optical elements, still common in US commercial and defense contracts.
• MIL-PRF-13830 — the military specification for optical components, which defines scratch-dig numbers (like 60-40 or 20-10) that many commercial shops still reference.

Each standard expresses tolerances differently. ISO 10110 specifies surface form as a power error in fringes (or waves) and irregularity as a fraction of a fringe. The scratch-dig notation from MIL-PRF-13830 quantifies surface defects: the first number is scratch width in microns (0.001 mm), the second is dig diameter in hundredths of a millimeter. A 60-40 scratch-dig spec allows a maximum scratch width of 0.06 mm and a maximum dig diameter of 0.40 mm.

Why This Matters

If your drawing uses ISO 10110 but your supplier inspects to MIL-PRF-13830, you're comparing apples to oranges. The standards define different measurement methods and different acceptance criteria. ISO 10110 also includes tolerance for centering errors (decentration and tilt) that MIL-PRF-13830 handles differently. A lens that passes one standard may fail the other.

Common Mistakes to Avoid

• Assuming scratch-dig equals surface quality: Scratch-dig only covers localized defects like scratches and pits. It says nothing about surface form (power and irregularity) or roughness. A lens can be 20-10 scratch-dig and still have terrible transmitted wavefront.
• Ignoring the ISO 10110 block format: Each block (3/, 4/, 5/) refers to a different tolerance type. If your drawing only shows a surface form tolerance without specifying the test wavelength, the measurement is ambiguous.
• Not specifying the test method: ISO 10110 allows several methods for measuring surface form, including interferometry and profilometry. The method affects the result.

Step 2 — Verify Surface Form with Interferometry

What to Do

Surface form — the deviation of the actual surface from the ideal spherical or aspheric shape — is measured with an interferometer. The instrument compares the test surface to a reference surface and produces an interference pattern. The pattern's fringe spacing corresponds to half the test wavelength.

For a typical HeNe laser interferometer operating at 632.8 nm, one fringe of power equals one-half wavelength of surface deviation, or approximately 0.316 microns. A tolerance of 3/ 2(0.5) in ISO 10110 notation means the power error is 2 fringes and the irregularity is 0.5 fringes.

The measurement procedure:

• Set up the interferometer with a reference flat or sphere matched to the test part's nominal radius.
• Align the test part to null the fringe pattern — this minimizes the number of fringes.
• Capture the interferogram and analyze it using phase-shifting software.
• Report power and irregularity separately, as they affect performance differently.

Why This Matters

Surface form errors directly affect wavefront quality. In a laser focusing system, a power error of one fringe (0.316 microns) can shift the focal length by a measurable amount. Irregularity — the asymmetric component of surface error — scatters light and degrades the point spread function. For diffraction-limited systems, you typically need surface form better than λ/10 (63 nm) peak-to-valley.

Common Mistakes to Avoid

• Confusing power with irregularity: Power is a spherical error that changes focal length. Irregularity is a nonspherical error that degrades image quality. They need separate tolerances.
• Measuring at the wrong wavelength: Some coatings are opaque at 632.8 nm. Use a test wavelength that the coating transmits, or measure the substrate before coating.
• Forgetting environmental stability: Interferometry is sensitive to vibration and air turbulence. Measurements taken on a shop floor without isolation can show false errors of λ/4 or worse.

Step 3 — Check Surface Roughness with Profilometry

What to Do

Surface roughness — the fine-scale texture on the optical surface — is measured with a contact profilometer or an atomic force microscope (AFM). The profilometer drags a diamond stylus across the surface and records the vertical displacement. The AFM uses a sharp tip to map the surface at nanometer resolution.

Typical roughness values for polished optical surfaces range from 0.5 nm Ra (root mean square) for precision optics to 5 nm Ra for standard components. The ISO 10110 standard specifies surface texture with the 5/ notation, expressed in units of roughness depth (Rt) or average roughness (Ra).

For most visible and near-infrared applications, a roughness of 1-2 nm Ra is sufficient. For UV applications or high-power laser optics, you may need 0.5 nm Ra or better. Roughness scatters light, and the scatter loss increases with the square of the roughness-to-wavelength ratio.

Why This Matters

Surface roughness affects three things: scatter loss, laser damage threshold, and coating adhesion. A rough surface scatters more light, which reduces throughput and creates stray light in imaging systems. In high-power lasers, rough surfaces concentrate electric fields and lower the damage threshold. The industry standard for laser optics is typically 1 nm Ra or better.

Common Mistakes to Avoid

• Using the wrong cutoff length: Profilometer measurements depend on the evaluation length and filter cutoff. A 0.8 mm cutoff gives different results than a 2.5 mm cutoff. Specify the measurement conditions.
• Measuring the wrong area: Roughness varies across the surface. Measure at multiple points, including the center and edges.
• Confusing Ra with Rq: Ra is the arithmetic average deviation; Rq is the root mean square deviation. For the same surface, Rq is typically 10-25% higher than Ra.

Step 4 — Verify Centration and Edge Geometry

What to Do

Centration error — the displacement and tilt of the optical axis relative to the mechanical axis — is measured with an autocollimator or a centration machine. The instrument projects a reticle through the lens and measures the reflected image position as the lens rotates.

For a typical precision lens, centration tolerance is 1-3 arcminutes. High-precision lenses for laser systems may require 5-10 arcseconds. The ISO 10110 standard specifies centering errors with the 4/ notation, which includes both decentration (lateral displacement) and tilt (angular error).

The measurement procedure:

• Mount the lens in a precision chuck or on a rotary stage.
• Align the autocollimator to the mechanical axis of rotation.
• Measure the runout of the reflected reticle image as the lens rotates 360 degrees.
• Calculate the centration error from the runout amplitude.

Why This Matters

Centration errors cause beam deviation and image shift. In a laser system, a centration error of 1 arcminute at a distance of 1 meter produces a beam displacement of approximately 0.29 mm. In an imaging system, centration errors cause coma and astigmatism that degrade resolution.

Common Mistakes to Avoid

• Measuring centration only at the center: Edge thickness variation (ETV) can cause centration errors that don't show up in a center-only measurement.
• Ignoring wedge: Wedge — the angle between the two lens surfaces — is a form of centration error. A wedge of 1 arcminute in a 10 mm diameter lens produces a beam deviation of about 0.0003 radians.
• Not checking the mechanical reference: The centration measurement is only as good as the mechanical reference. If the chuck is dirty or the lens is not seated properly, the measurement is meaningless.

Step 5 — Audit the Supplier's Inspection Report

What to Do

A proper inspection report includes measured values, not just a pass/fail stamp. Review the report for:

• Measured values for each tolerance: Power, irregularity, roughness, centration, and scratch-dig.
• Test method and equipment: Which interferometer, profilometer, or autocollimator was used?
• Environmental conditions: Temperature and humidity during measurement.
• Test wavelength: For interferometric measurements, the wavelength matters.
• Sample size: Was every part measured, or only a sample?

For high-volume production, suppliers often use 100% inspection for critical parameters and statistical sampling for others. AQL (Acceptable Quality Limit) sampling plans, such as those in ISO 2859, specify how many parts to inspect based on lot size and quality level.

Why This Matters

The inspection report is your only evidence that the parts meet specification. If the report lacks measured values, you cannot verify compliance. If the test method is not stated, you cannot reproduce the measurement. A supplier that provides detailed inspection data demonstrates confidence in their process.

Common Mistakes to Avoid

• Accepting a certificate of conformance without data: A COC only states that parts passed inspection. It doesn't tell you what was measured or how.
• Not specifying inspection requirements in the PO: If you need measured data, put it in the purchase order. Otherwise, you may get parts with no inspection report.
• Ignoring measurement uncertainty: Every measurement has uncertainty. A tolerance of 0.5 fringes with a measurement uncertainty of 0.2 fringes leaves little margin. Ask about the supplier's measurement capability.

Pro Tips for Success

• Request a sample inspection before production: Send your drawing to the supplier and ask for a pre-production sample with full inspection data. This verifies their capability before you commit to a large order.
• Specify the test wavelength in your drawing: If you don't, the supplier will use their standard wavelength, which may not match your application.
• For high-power laser optics, require laser damage testing: Surface quality and roughness alone don't guarantee damage resistance. Ask for damage threshold data per ISO 21254.
• Consider the coating's effect on tolerances: Coatings add stress that can deform the substrate. For precision optics, specify the tolerance after coating, not before.
• Build relationships with suppliers who understand your application: For demanding applications like Laser Optical System s, a supplier who knows your requirements will flag potential issues before they become problems. Similarly, Medical Instruments demand tight tolerances and traceable inspection data — make sure your supplier can provide both.

Frequently Asked Questions

What is the difference between ISO 10110 and MIL-PRF-13830?

ISO 10110 is an international standard that uses a block notation system for surface form, texture, and centering tolerances. MIL-PRF-13830 is a US military specification that uses scratch-dig notation for surface defects. ISO 10110 provides more comprehensive tolerance definitions, while MIL-PRF-13830 is simpler but less complete.

How many fringes of power error are acceptable for a precision lens?

For most precision applications, a power error of 1-2 fringes (0.316-0.632 microns at 632.8 nm) is acceptable. Diffraction-limited systems typically require λ/10 (0.063 microns) or better. The acceptable value depends on your application's wavefront error budget.

Can surface roughness be measured without contacting the surface?

Yes. Optical profilometry using white light interferometry or confocal microscopy can measure roughness without contact. These methods are faster than contact profilometry and don't risk scratching the surface, but they have different spatial resolution limits.

What does a 60-40 scratch-dig specification mean?

The first number (60) is the maximum scratch width in microns (0.06 mm). The second number (40) is the maximum dig diameter in hundredths of a millimeter (0.40 mm). This is a moderate surface quality spec, suitable for most commercial applications.

How do I verify that a supplier's inspection data is accurate?

Request the measurement method, equipment model, and environmental conditions for each measurement. If possible, send a reference part with known characteristics for the supplier to measure. Compare their results to your own measurements or to an independent lab.

Conclusion

Tolerancing standards and metrology methods used to verify custom optical lens quality are not optional paperwork — they determine whether your system performs as designed. By understanding ISO 10110, MIL-PRF-13830, and the measurement techniques behind each tolerance, you can specify parts correctly, audit supplier inspection reports, and avoid costly surprises.

Start by reviewing your drawing's tolerance callouts. Verify that your supplier uses the same standards and test methods you expect. Request measured data, not just certificates. And for critical applications, require pre-production samples with full inspection reports.

The industry standards for optical tolerancing — ISO 10110, ANSI/OEOSC, and MIL-PRF-13830 — provide a common language for specifying and verifying optical quality. Learn that language, and you'll never again wonder whether the lenses you received actually meet the spec. Your system will perform as designed, your suppliers will respect your technical competence, and your projects will stay on schedule. That's the payoff for investing an hour in understanding how optical quality is measured. Relevant specifications and application guidance are available through Medical Instruments.