Fix actor map: Multiple instances of same file

Created on Wednesday 12 May 2021, 06:21

Back to task list
  • ID
    987310
  • Project
    CityLoops
  • Status
    Completed
  • Priority
    Medium
  • Type
    Programming work
  • Assigned to
    Carolin Bellstedt
  • Subscribers
    Carolin Bellstedt
    Johanna Järvinen
    Paul Hoekman

You are not logged in

Log In Register

Please join us and let's build things, together!

Description

Something weird is happening on the master map: It displays several instances of the same file in the legend, as well as in the table below, see the one of Mikkeli, for example. There are 12 instances of waste treatment, landfill, because of this file "Geo-localisation of waste collecting and treatment in Mikkeli", but also 2 of the Road network in Mikkeli, see under Land use > Roads.

I was thinking that it could have something to do with the tags. Guus tested something locally (he will comment), but it didn't remove all instances.

Discussion and updates


New task was created


Task was assigned to Paul Hoekman


Guus, I see the following code on line 521 of views.py in cityloops:

list = LibraryItem.objects.filter(spaces=space, meta_data__processed__isnull=False, type_id__in=[40,41], tags__in=[975,976,977,978,979]) | LibraryItem.objects.filter(spaces=space, meta_data__processed__isnull=False, type_id__in=[40,41], tags__in=[996,997,1080,1000,1001,1002,1003,1004,1005,1006,1007,1008,1009,1010,1041]).order_by("date_created")

Seems like you are adding two different querysets, both with exactly the same initial query, but with two different lists of tags. What exactly is happening there? I'm pretty sure we gotta change that query but before I do can you elaborate on the double query here?


Paul I added that for this ticket: https://cityloops.metabolismofcities.org/hub/work/982540/

I just pushed something that fixes the query above (line 543). Please check if that's OK.

I did notice a data error:

https://cityloops.metabolismofcities.org/library/980987/

and

https://cityloops.metabolismofcities.org/library/985693/

are the same file, but with different tags and uploaded twice. So I think one needs to be removed.

I think that's part of the problem. But another seems to be that a new entry in the layer menu (on the right, to add/remove layers) is added for the same layer when they have multiple tags. So the "waste collecting and treatment" layer is added a ton of times because it has many tags. I removed this file locally to see what would happen, but that somehow also removed the duplicate file.

I noticed two more files that are identical:

https://cityloops.metabolismofcities.org/library/767593/ and https://cityloops.metabolismofcities.org/library/766719/.

So yeah, I think part of the problem was my mistake in the code, but another is the duplicate files.


Hi Guus,

Ah OK noted, great. The fixed code looks good indeed. A single condition plus distinct() was what I had in mind there as well.

Carolin can you look into the duplicated files and get them sorted out? Once this is cleaned up we can see if everything is in order.


Paul Hoekman is no longer responsible for this task


Hi Paul,
Yes, thanks, I will look into those.


Task was assigned to Carolin Bellstedt


Status change: Open → In Progress


Ok, so what I've gathered is:
1) The two waste files: A) Geo-localisation of waste collecting and treatment in Mikkeli has 11 items, whereas B) GPS coordinates to waste collecting and treatment in Mikkeli 29.4 has 8 items on the map, see screenshot.

2) The two municipal divisions: A) Mikkeli Municipal Division 1:100 000 and B) Municipal division of Mikkeli, which is in scale of 1:10 000. - These actually don't pose a problem, because only the 2A is processed at the moment and there is probably no need to process 2B.

@Johanna, is there added value in having two files? If not, could you please delete the less relevant file of the waste actors 1A or 1B? I don't know if the additional 3 actors play are role or why they are missing in the other file. Alternatively you could also split them up into collection and treatment, but that is up to you. Thanks!


Hi Carolin. I deleted the first one since it had the sludge collectors as well. I had downloaded that file first and then realized that I didn´t want the sludge collectors in this entry at all so then I asked you to delete it inorder to upload the new file without them. I hope everything is ok now that I deleted the unwanted file.


Hi Johanna,
Thanks for sorting this out. Yes, I remember the situation with the sludge collectors and everything is ok now :)

@Paul, everything is good here now from the original several instances issues. Something else I've just noticed is that the NUTS3 boundaries don't show, though this isn't a Mikkeli specific issue. My guess is that is has to do it with the NUTS3 being so many items. Anyway, this is not relevant for now.

Marking this as completed then. Thanks everyone!


Status change: In Progress → Completed


Perfect, glad to hear!