Choropleth doesn't bind if using numeric index

[alessioarena]

Please add a code sample or a nbviewer link, copy-pastable if possible

gdf = gpd.read_file('SA2 shapefile')
gdf.columns
# output: ['name', 'probability']
gdf.index.dtype
# output: dtype('int64')

m = folium.Map(location=[	-37, 144], zoom_start=7, tiles='cartodbpositron')
folium.Choropleth(
    geo_data=gdf.geometry,
    data= gdf.probability,
    key_on='feature.id',
    fill_color='YlGn',
    fill_opacity=0.7,
    line_opacity=0.2,
    nan_fill_color= 'purple',
    legend_name='Probability',
    highlight=True
).add_to(m)

Problem description

I'm pretty new to folium, but I did work with geopandas and pandas a lot.
I also cannot provide the data, but that should be reproducible with any shapefile if the problem is what I think is.

My understanding of the code above is that will try to:

• select the geometry column and pass it as a GeoSeries for conversion to GeoJSON
• select the probability column and pass it as a Series
• leveraging the common index, trying to bind the two on feature.id <-> index
• do some magic and spit the choropleth out

Unfortunately this is not working as expected, as no error was raised but all polygons have a nice purple color (NaN). The legend was correctly generated so I reckon is purely a merging/binding issue.

Doing some quick troubleshooting I suspect that the bind failed because the feature.id is now a string, while the Series index is numeric, but I may be very wrong.

This instead worked as expected:

folium.Choropleth(
    geo_data=gdf,
    data= gdf,
    columns=['name', 'probability'],
    key_on='feature.properties.name',
    fill_color='YlGn',
    fill_opacity=0.7,
    line_opacity=0.2,
    nan_fill_color= 'purple',
    legend_name='Probability',
    highlight=True
).add_to(m)

Happy to do additional troubleshooting if needed.

Expected Output

A nice colorful choropleth without NaNs (they were dropped at a previous step)

Output of folium.__version__

'0.9.1'

[Conengmo]

Hi @alessioarena, thanks for your report. I think you're right, if the index keys are integer and the feature ids are string, it won't match. I think here's the line where that goes wrong:

https://github.com/python-visualization/folium/blob/master/folium/features.py#L1115

Here color_data is a dict with the Series index as keys. And key_of_x is probably a string from the geojson feature id field. Maybe you can check if that's indeed the case?

What would be the best way to take action on this? I'm thinking:

• Add a warning when none of the keys match.
• If color_data has int keys, try the string version as well in the lookup.
• Auto-convert color_data keys from int to str if all key_on values are str.
• More documentation.

Any more ideas? Do you have an opinion on this? I don't know if you have time to help with a PR but that would be welcome.

[alessioarena]

Yes, I should be able to help with this.
Is there any case where we should not coerce color_data keys to string? Or seen from the other direction, is there any case where key_of_x is not a string?

I can do the required investigations, but I'm just wondering if you can think of specific case where this occurs.

[Conengmo]

Yes, key_on can refer to any field in a geojson feature. It can also be an integer for example.

We actually just fixed an issue where the geojson feature value was an integer, which we didn't handle well: #1169

[alessioarena]

I made a fix for this behaviour and is sitting in my forked repository

This addresses your second and third suggestion, but I'm unsure how to tackle the warning.
In fact the join is done individually for each feature in the GeoJSON, and at that point we would not know if the NaNs are caused by incorrect join or simply empty data.

If someone has a suggestion I'm happy to follow it, otherwise I will issue a PR and go from there.

Re the fourth point, at the moment I made 3 new tests where geometries are pulled from a GeoDataFrame (that is where the conversion to str for the index takes place).
The data is instead coming from:

1. the GeoDataFrame, where index is of type int and data is int
2. a DataFrame where index is object (int/str mix) and data is int
3. a DataFrame where indes is str and data is int (the only working case with old behaviour)

[Conengmo]

Hi! Thanks for working on this. On first glance it looks good, though I’ll have to think about it a bit more. Can you open a PR? I’ll review in the PR. I’m on vacation now so it’ll be after next week.

[Conengmo]

Should be fixed with #1193, which will be included in the upcoming 0.14.0 release.