New task was created
Implement queries so all data from a dataset is converted into a single unit
Created on Thursday 1 September 2022, 13:05Back to task list
ProjectMetabolism of Cities Data Hub
Assigned toNo one yet
SubscribersCarolin BellstedtPaul Hoekmanjoe
Objective: With the data, we are often dealing with many different units. (See list of units here.) It would save the user time, if they could upload data with varying units of the same type (e.g. volume or weight), for example of mass units in kg, tons, and short tons. The objective is that the system converts the secondary units, using the factors that exist for them to the primary unit OR that all units are converted to the same unit, using the factors. This is needed so that for the visualizations, they are all in a single unit. It will also be a useful addition to the site, which is most handy when combining data from multiple datasets into a single visualization.
Audience: This function is for someone who uploads a lot of data or is actually submitting processed data, ready to be read by the system. To save time doing manual conversions in Excel/LibreOffice etc. they ensure that the units are of the same type, but don't have to bother with factors etc.
How it works: Using the example of a dataset with three varying weight units, to test for the processing linked here and attached. (The original file is here. The first three materials in the table should look like the visualisation in the end.), let's see how this should work.
- Following the processing of flows or stocks queue to process the file, the user starts the process.
- In the "review of file content" stage (stage 2 out of 3), the material and reference spaces are classified, see link. So far, there is no visible "unit check" where the system confirms what the unit is. The file processing can be finalised, regardless if the units are the same or not.
- I imagine, the unit check could already take place at this point, and not just when doing the data visualisation. Here the intervention could be that the system reads the unit column and then lists all units that were identified with the other two classification boxes, see screenshot. As opposed to the material check, it should not only analyse the first 10 records, but all.
- The user should confirm that these units are all of the same type. There could be a link to the units (https://staf.metabolismofcities.org/units/) and a dropdown or list, where the user selects that e.g. the unit type is WEIGHT (in our example).
- The user should then be asked to choose with which unit they would like to proceed, suggesting the primary unit (tonnes, in our case).
- The system should convert the values that are not the target unit with the respective factors. We already have the system to convert units, but the query must be changed so that this conversion is done when extracting the data from the db.