Upload FAO production data

Created on Tuesday 20 April 2021, 07:15

Back to task list
  • ID
    982990
  • Project
    Metabolism of Cities
  • Status
    In Progress
  • Priority
    Medium
  • Type
    Upload Data
  • Tags
    Data Hub Priority Plan 2021 MFA integration
  • Assigned to
    Taina Wanderley
  • Subscribers
    Aristide Athanassiadis
    Francisco Felix
    Paul Hoekman
    Taina Wanderley

You are not logged in

Log In Register

Please join us and let's build things, together!

Description

Given that we can't have only one dataset for extraction flows for the entire world (IRP) we will need to do it piece by piece. FAO produces data on crops, forestry and livestock production, imports and exports.

Go to action page

Discussion and updates


New task was created


Task was assigned to Taina Wanderley


Status change: Open → In Progress


So... just to confirm "piece by piece" means to load this data for each city already activated here? For ex, I got to Geneva, "stocks and flows" and in "Extraction: Agriculture" I put this data from FAO for Switzerland.
The CSV format is good, right?


ok, this is part of the multi-scale integration that we're talking about. So, maybe there will be a country-level sub-site?


Hi Taina,

In this case piece-by-piece refers to each of the flows (extraction -- imports -- exports -- etc). Each of them needs to be uploaded separately instead of us being able to load an entire MFA dataset that contains all information at once (the IRP dataset, which unfortunately does not come with an open license).

So you can load ALL info for extraction that is available at once. Where exactly to put it we can discuss later today in the sprint.


Tks Paul!
We'll discuss later then.
see you soon


Hi Taina, just to recap from our meeting: you can use Munich as the place to dump ALL data for multiple scales such as global datasets on all cities or countries.

Secondly, please note I have activated Brasília here.


Hey, Paul, thank you! I'll work on it :)


Hi, Paul!
I'm progressing in this task, with this 2 uploads completed:

FAOSTAT Crops Data of all countries
FAOSTAT Forestry Production and Trade Data of all countries

Everything is being registered in the city of Munich, as agreed. As I'm having some issues with the files, I ask that if anyone wants to check it out to see if everything is alright.... from what I've seen, these first two items are :)

BUT, when I start a third item "FAOSTAT Detailed trade matrix Data (imports/exports) of all countries", I got the following error " 413 Request Entity Too Large" and I couldn't load the files here.
I tried a few things but nothing worked.
Any idea how I do it, Paul?


Hi Taina,

Great work already! I will check those out over the weekend and let's chat on Monday's sprint if you are there. For the error that you get... how large is the file you are trying to upload? Sounds like the file size is an issue.


okay, yes, I'll be in the meeting next Monday; if I can not do it, I'll let you know :)
Yes, this is clearly a file size issue haha ; 1,32 GB
Maybe for this one I could upload the zip file? (I didn't do it, because I remember you saying to not)

Last edited: 2021-06-18 17:57:53.698311+00:00