The FHIR standard embeds floats with precision as numbers within JSON despite the fact that, to do so, computers in general and the JSON standard in particular demand that you represent such numbers as strings.
Doing this complicates and breaks numerous libraries, can cause issues with precision, and is highly problematic for just about any programmer dealing with this standard.
Why oh why did you do this? Can you please change it?
In healthcare, it’s essential to retain precision of numeric information. 3.100 is a distinct number (and behaves differently in calculations) than 3.1. The requirement to retain precision information trumped alignment with common JSON convention.
JSON standards for preserving numeric precision are to encode numbers as strings…
It could literally cause issues that kill people – as there are underlying systems that will mutate the data: https://github.com/michalmuskala/jason/issues/71#issuecomment-478602170
Did you read the comment here? http://hl7.org/fhir/json.html#primitive
As for literally killing people… we discussed this at length, with participation from metrologists (I was one in the past). I think you won’t find relevant clinical scenarios where the difference between 9007199254740993.0 and 9007199254740992.0 (or similar) is life critical.
As for changing it: that is impossible under our procedural rules
I did read all of it. The thing is, FHIR is violating a basic standard of the Web – and not even following the parse method embedded in standard browsers. That’s simply terrible.
How can one promote a standard while hypocritically trashing a much, much larger standard employed by many other industries and consumers?
And the procedural rules won’t let you fix it???
This all sounds rather crazy from a person who’s been programming professionally for 20+ years.
I feel like this will become a larger issue long-term - both as interop is desired between standards and more developers work with FHIR.
Anyone working with any other system producing or consuming JSON will need to separate out the parsing and generation of data between “standard” and “FHIR compliant”. Developers will then need to remember which one to use on a given data set at a given time, or else translation errors like loss of precision will occur.
The simplest case I can see would be a value similar to:
{ “val” : 3.0000000000 }
A standard parser will treat that as an integer value and reduce to a single significant digit - without warning or error.
For a trivial use-case, consider a web API that modifies JSON as data flows through it (e.g., adding a value for tracking) by parsing and recreating the object.
In the example you post @grahamegrieve , the relative error is very small and is not likely to be clinically relevant. As @lloyd said above, precision errors are quite likely to be clinically relevant.
The current solution of modified JSON works as long as a developer stays within the FHIR bubble and is aware of this behavior. Given that we refer to the data as JSON, developers will (rightfully) expect JSON-formatted data. Further given that standard parsers will work, people may not discover an issue until an incident has occurred.
Thoughts?
Thanks!
-Gino
Thanks for voicing my concerns in a more professional manner! I was stridently upset when I posted the other night and acted the jerk.
It’s just hard for me to understand how, when the XML variant simply encodes all numbers as strings – and there are so many extant versions of FHIR, that breaking the JSON standard could be anything other than an act of intentional sabotage similar to regulatory capture (adding useless barriers to prevent competition in a space).
Thanks!
There is a valid point here but @scott you have to remember that if changed, 99% of the developers will be going “wtf” as to why numbers are represented as strings, because nobody wants to deal with that either.
Are you working with a use-case where this is clinically relevant?
You’re not making friends with claims like that either. Please see our Code of Conduct at http://wiki.hl7.org/index.php?title=FHIR_Code_of_Conduct
FHIR is an open standard and community. We’re 7 years into our development and 100s of developers have implemented this, and the text I referred you to was added many years ago. Anyone has been free to comment on it, and no one has. See https://chat.fhir.org/#narrow/stream/179166-implementers/topic/JSON.20decimals for further details.
@grahamegrieve I’m sorry, I should’ve worded that in a less accusatory way. I was trying to emphasize that it was “hard for me to understand how” such a thing came to be.
@VadimPeretokin every developer with a browser hitting the XML service already receives all numbers as strings. Anyone mapping the data with a library against the spec would have something that could auto-convert them from string to a numeric value with an associated precision without abandoning standard encoders/decoders for the JSON spec.
Some background:
I’ve been building a FHIR library that uses the provided XSDs as a code generation source.
The biggest issues I’ve encountered thus far:
There’s no version number in FHIR resources (as there was in HL7). Our integration system will be handling multiple versions of FHIR concurrently due to thousands of existing vendor connections.
I’ve had to create a special data structure to facilitate Decimal encoding – and fork the standard JSON decoder to handle decimal types.
Technically, I have it working.
My concerns are mostly that deviations from widely used standards could
result in medical errors if someone writes an ad-hoc REST JSON in the browser or another stack and loses precision for a field in which it matters.
means that highly optimized, secure and trusted standard libraries (like JSON.parse) have to be avoided or reimplemented in any technology stack attempting to work in this space.
Besides a simple string substitution, the float could be provided with an accompany precision integer – but, at this point, that seems like a much larger change.
Yes if implementers fail to follow the specification then that could potentially result in patient safety problems. However that’s really out of scope for the standard, a no different from the situation that existed with HL7 V2 and V3.