csv to raster python

Question

3 column csv (Lon, Lat, Ref) (63000 rows) and I would like to convert the "Ref" to raster. points (x,y) are being plotted. I want to plot the "Ref" column and add contour and color-fill it. Thanks

Data:

Lon,Lat, Ref
-115.0377,51.9147,0
-115.0679,51.9237,0
-115.0528,51.9237,0
-115.0377,51.9237,0
-115.1134,51.9416,0
-115.0982,51.9416,0
-115.0831,51.9416,0
-115.1437,51.9596,6
-115.1285,51.9596,6
-115.1588,51.9686,6
-115.1437,51.9686,10.5
-115.1285,51.9686,10.5
-115.1134,51.9686,8
-115.1891,51.9776,7.5
-115.174,51.9776,7.5
-115.1588,51.9776,7.5
-115.1437,51.9776,8
-115.1285,51.9776,8
-115.1134,51.9776,8
-115.1891,51.9866,7
-115.174,51.9866,7
-115.1588,51.9866,7
-115.1437,51.9866,0
-115.1285,51.9866,0
-115.1134,51.9866,0
-115.1891,51.9956,7
-113.1143,52.2385,3.5
-113.0992,52.2475,3.5
-113.084,52.2475,3.5
-113.0689,52.2475,5.5
-113.0537,52.2475,5.5

Code:

import pandas as pd
import geopandas
from shapely.geometry import Point
import fiona
import matplotlib.pyplot as plt

df=pd.read_csv('name.csv')
df1=df.interpolate()

geometry=[Point(xyz) for xyz in zip(df1.ix[:,0], df1.ix[:,1], df1.ix[:,2])]

df3=geopandas.GeoDataFrame(df1, geometry=geometry)

df3.plot()

plt.savefig('raster.tiff')

wanted result:

by plot, i mean to convert to raster (color)

user2031063
– user2031063

2018-12-19 15:48:33 +00:00
Commented Dec 19, 2018 at 15:48 — user2031063
– user2031063, Commented Dec 19, 2018 at 15:48

Ahresse · Accepted Answer · 2020-07-12 09:46:29Z

1

If you want to plot points from GeoPandas based on the "Ref" column, you don't need it as a z coordinate.

import pandas as pd
import geopandas
from shapely.geometry import Point
import matplotlib.pyplot as plt

df = pd.read_csv('name.csv')

geometry = [Point(xy) for xy in zip(df.iloc[:, 0], df.iloc[:, 1])]

gdf = geopandas.GeoDataFrame(df, geometry=geometry)

gdf.plot(column=' Ref')

plt.savefig('raster.tiff')

You don't even need interpolate(). However, if you want to convert your vector point dataset to raster geoTIFF, plot() is not the right way to do it. I would go for gdal.Grid() as explained here. - [Python - gdal.Grid() correct use][1]

EDIT Using gdal.Grid() like this I am able to generate tif based on the sample of data you provided.

import os
import gdal

dir_with_csvs = r"/home/panda"
os.chdir(dir_with_csvs)

def find_csv_filenames(path_to_dir, suffix=".csv"):
    filenames = os.listdir(path_to_dir)
    return [ filename for filename in filenames if filename.endswith(suffix) ]
csvfiles = find_csv_filenames(dir_with_csvs)
for fn in csvfiles:
    vrt_fn = fn.replace(".csv", ".vrt")
    lyr_name = fn.replace('.csv', '')
    out_tif = fn.replace('.csv', '.tiff')
    with open(vrt_fn, 'w') as fn_vrt:
        fn_vrt.write('<OGRVRTDataSource>\n')
        fn_vrt.write('\t<OGRVRTLayer name="%s">\n' % lyr_name)
        fn_vrt.write('\t\t<SrcDataSource>%s</SrcDataSource>\n' % fn)
        fn_vrt.write('\t\t<GeometryType>wkbPoint</GeometryType>\n')
        fn_vrt.write('\t\t<GeometryField encoding="PointFromColumns" x="Lon" y="Lat" z="Ref"/>\n')
        fn_vrt.write('\t</OGRVRTLayer>\n')
        fn_vrt.write('</OGRVRTDataSource>\n')

output = gdal.Grid('outcome.tif','name.vrt')
# below using your settings - I don't have sample large enough to properly test it, but it is generating file as well  
output2 = gdal.Grid('outcome2.tif','name.vrt', algorithm='invdist:power=2.0:smoothing=1.0')

Do you have any particular reason to use gdal via shell? [1]: https://gis.stackexchange.com/questions/254330/python-gdal-grid-correct-use

edited Jul 12, 2020 at 9:46

Ahresse

6057 silver badges21 bronze badges

answered Dec 19, 2018 at 16:58

martinfleis

7,8942 gold badges29 silver badges37 bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

martinfleis Over a year ago

I've just added version adapting your goal script which works for me.

user2031063 Over a year ago

thank. no reference to shell. I copied your code and it is showing an error: Traceback (most recent call last): File "whkout3.py", line 26, in <module> gdal.Grid('outcome2.tif','name.vrt', algorithm='invdist:power=2.0:smoothing=1.0') File "/usr/lib/python2.7/dist-packages/osgeo/gdal.py", line 973, in Grid return GridInternal(destName, srcDS, opts, callback, callback_data) File "/usr/lib/python2.7/dist-packages/osgeo/gdal.py", line 3220, in GridInternal return _gdal.GridInternal(*args) ValueError: Received a NULL pointer.

user2031063 Over a year ago

same error with the first simpler output. Thanks for your help

martinfleis Over a year ago

Looking at your wanted result, I've got question - do you want geo tiff with 'Ref' as values to use it as a raster spatial data layer or do you want just that colourful output? If B, you can try to plot it via GeoPandas with marker='s', markersize=[your grid size]. I am unable to replicate your error, but I see you are using python 2.7, so if your deserved output is geo tiff, I'll try to set env for that and will see.

user2031063 Over a year ago

I want both (Ref values as a raster spatial data layer and the colorful output. I was reading about my error, though not conclusive but it could be related to python/gdal version. i have the latest gdal. Thank you help.

|

user2031063 · Accepted Answer · 2018-12-19 18:48:52Z

@ctvtkar, I am attaching a code here using gdal. When I run it, the file.vrt gets created, but not the .tif file. The erroe i get is: gdal_grid: not found. gdal is instaled

Code:

import subprocess
import os

dir_with_csvs = r"/home/panda"
os.chdir(dir_with_csvs)

def find_csv_filenames(path_to_dir, suffix=".csv"):
    filenames = os.listdir(path_to_dir)
    return [ filename for filename in filenames if filename.endswith(suffix) ]
csvfiles = find_csv_filenames(dir_with_csvs)
for fn in csvfiles:
    vrt_fn = fn.replace(".csv", ".vrt")
    lyr_name = fn.replace('.csv', '')
    out_tif = fn.replace('.csv', '.tiff')
    with open(vrt_fn, 'w') as fn_vrt:
        fn_vrt.write('<OGRVRTDataSource>\n')
        fn_vrt.write('\t<OGRVRTLayer name="%s">\n' % lyr_name)
        fn_vrt.write('\t\t<SrcDataSource>%s</SrcDataSource>\n' % fn)
        fn_vrt.write('\t\t<GeometryType>wkbPoint</GeometryType>\n')
        fn_vrt.write('\t\t<GeometryField encoding="PointFromColumns" x="Lon" y="Lat" z="Ref"/>\n')
        fn_vrt.write('\t</OGRVRTLayer>\n')
        fn_vrt.write('</OGRVRTDataSource>\n')

    gdal_cmd = 'gdal_grid -a invdist:power=2.0:smoothing=1.0 -zfield "Ref" -of GTiff -ot Float64 -l %s %s %s' % (lyr_name, vrt_fn, out_tif)

    subprocess.call(gdal_cmd, shell=True)

Collectives™ on Stack Overflow

csv to raster python

2 Answers 2

8 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

8 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related