Discuss and decide + write rules for importing of Eurostat datasets

Created on Thursday 11 February 2021, 09:48

Back to task list
  • ID
    767726
  • Project
    CityLoops
  • Status
    In Progress
  • Priority
    High
  • Type
    General Research Work
  • Assigned to
    Carolin Bellstedt
  • Subscribers
    Aristide Athanassiadis
    Carolin Bellstedt
    Paul Hoekman

You are not logged in

Log In Register

Please join us and let's build things, together!

Description

We have our great Eurostat manager and have invested lots of time to go through most of the 9949 Eurostat datasets, 472 are still pending, to decide if they are relevant from a UM/CL perspective or not.

Now we are in the situation, where the CL cities who find those datasets too, will add the same one, on their own. It could be that they all add "Gross domestic product (GDP) at current market prices by NUTS 3 regions" (especially because there is no name check for a dataset, when adding one, which normally makes sense). Do we want to have 7 instances of the same dataset? Probably not, right? So we should decide rather sooner than later, if we do import the Eurostat datasets or instead see what the cities will actually do.

Of course, there is no need to import all 9,949 Eurostat records, but we could discuss if it makes sense to import if not the
800 accepted records, then perhaps the 155 accepted NUTS records:

  • 155 accepted NUTS, https://cityloops.metabolismofcities.org/eurostat/?q=nuts&show=accepted
  • 155 is made up of 40 accepted NUTS 3 (incl. NUTS3) https://cityloops.metabolismofcities.org/eurostat/?q=nuts+3&show=accepted
  • and 114 accepted NUTS 2 (plus 1 NUTS2) https://cityloops.metabolismofcities.org/eurostat/?q=nuts+2&show=accepted

We will again run into the issue of having MoC and CL tags, there is no way around it, but for now we could use CL tags and have the cities tag them. OR we add the CL tags in addition and the cities as spaces, which is possible in the backend, as I've just tested it.

In any case, first we should decide: to import or not to import? Aris, Paul, what do you think?

General instructions

Compiling lists, searching for information or compiling new information to further the goals of Metabolism of Cities

Discussion and updates


New task was created


My feeling: yes absolutely let's import them. Lots of advantages, no big drawback. I'd say let's do all 800 records. At least just having them in there is very useful. And we should look into some semi-automated scripts to label certain cities/reference spaces that are relevant. If you can put the rules in writing then I am happy to write a script for that.


Task was assigned to Carolin Bellstedt


Ok, cool!! Thanks! I will put the rules in writing and let you know when I have those done.


Status change: Open → In Progress


I've written up a document on Eurostat datasets, manager, grid and import to Data Hub, because we had a lot of info in different places. This document contains a section that lists points that we should address before the import, namely



  1. What should we do with the 472 pending ones? I suppose they could be imported later with the same script?
  2. What do we do with the info in the notes field?
  3. Check with Paul if assigning the correct ref spaces is part of the import? (point 9-12 in the “rules”)


The document also has some rules. Paul, could you please have a look and let me know if this is what you had in mind? Thanks!


Great Carolin, thanks. Quick feedback:



  1. Yes indeed
  2. Add it to the system under UPLOADER COMMENTS, which will then show on the page of the item in the library
  3. Hmmm I hadn't planned on that.............


Rules look good, had a quick look. Is there any deadline for this?


Thanks for your comments Paul.
1. Cool.
2. Nice.
3. Ok. Let's not do it then. Some of the ref spaces aren't in the system yet anyway and we haven't figured out yet how we go about naming of ref spaces in the case where there are several spatial units with the same name, but different codes (this whole City (C), Greater City (K), Functional Urban Area (L) business), see e.g. Dublin , https://ec.europa.eu/eurostat/web/cities/spatial-units

Would be great to have by next week, morning of Friday (Feb 26) for Aris to make a video on it. Doable?


Copy that Carolin. But it starts to feel a bit too much like work work with all the upcoming deadlines. Can I instead try to have this done in two weeks time, can you work with that? Safest would be to say March 7 so I also have the weekend if need be.


Yeah, I understand that. I've been trying to keep it easy, but it is a push on several fronts right now, which makes it tricky. I don't want it to feel like work work for you.
Yes, we will work with that then. Thanks!