In Julia at least, there are packages that provide alternatives to Bigfloats:

Logarithmic numbers: more range, less precision https://github.com/cjdoris/LogarithmicNumbers.jl

Double arithmetic: stitch 2 floats together https://github.com/JuliaMath/DoubleFloats.jl

Of course, if you are calling BLAS/LAPACK, you are constrained to use floats, but the recommendation on DoubleFloats is clear: if you know you algorithms, use the increased precision only in the parts that matter