Updated August 23, 2021 to reflect new tools added to the Agriculture Extension.
Last year, John Lindsay and I collaborated on a project at the University of Guelph within the Precision Agriculture sector that involved studying the relation between topographic attributes and crop yield. In completing this work, we were given many yield data sets for various farms and crop types. For those that have worked with crop yield data previously, you are undoubtedly aware of how noisy these data sets can be. Crop yield estimates that are measured using combine harvesters commonly suffer from high noise levels.
Yield data are so noisy because of the way that they are collected. For example, where harvested rows overlap, lower-than-expected crop yields may be measured within the second overlapping swath because the head of the harvester is only partially filled. Field edges are particularly susceptible to being harvested without a full swath of crop, resulting in anomalous crop yields. The starts of new swaths are also prone to errors, because of the misalignment between the time when the monitor begins recording and the time when grain begins flowing. Sudden changes in harvester speed, either speeding up or slowing down, can also result in errors in yield measurements.
Noisy crop yield data is a good example of an everyday problem that GIS professionals working in the agriculture sector encounter. Not cleaning these erroneous data points from your raw crop yield data can have negative implications the derived products of yield data. Many scientific studies stress the importance of cleaning yield data prior to analysis.
After conducting a literature review on best practices for cleaning and removing erroneous yield point data, I found numerous ways that the problem could be handled. However, surprisingly it seems that many professionals in this area carry out manual data cleaning, commonly using Excel or R Studio.
The Precision Agriculture Toolbox
After experiencing this issue first hand, John and I decided that it might be best to create a Precision Agriculture Toolbox, which contained tools that were able to overcome these common and sometimes complex problems that GIS practitioners, academics and professionals have everyday when working in the field of precision agriculture. Currently, six tools comprise this toolbox: The YieldFilter, YieldMap, YieldNormalization, ReconcileMultipleHeaders, RemoveFieldEdgePoints, and RecreatePassLines tool. All six of these tools are apart of of the Whitebox Agriculture Extension and the Whitebox General Toolset extension, which are among our first pay-for-use software packages aimed at providing support for the development of WhiteboxTools.
The YieldFilter tool can smooth yield point patterns, particularly accounting for difference among adjacent swath lines. This tool works effectively for crop fields of all sizes and is highly efficient. The image below is an example of yield data for a field in Ontario, Canada. There are 24,943 yield points in this field. The crop type is corn and this is the yield data straight off of the combine. The range of yield is 6 – 1688 bushels/acre where green represents lower yield and red depicts higher values. The YieldFilter tool took 0.5 seconds to execute on this dataset.
This is an example of how the YieldFilter tool could be set up within the WhiteboxTools environment. Shown here is our WB_Runner.py GUI. Be sure to set the value of your input file, the Yield Field name, Pass Field name and the output points. The swath width parameters comes with a default of 6.096 m (20 ft) as this is a common swath width. However, it could be changed accordingly.
How do you get the Swath Width from the yield points? I use the Measure Tool in QGIS to measure the distance from one yield point to another, as a rough estimate of the combine swath width. Here is an example.
The image below shows the result of the applying the tool to the data using the following parameters: swath width = 6.096m, z-score = 2.5, min. yield = 30.0, max. yield = 300.0.
The YieldFilter Tool is an extremely efficient tool for cleaning yield data. This tool is not limited to a certain crop type and is extremely customizable for your workflow. Using this tool will save you and your business time and money and will also provide you with a more accurate representation of in-field yield patterns. Check it out, along with the other 18 tools that make up the Whitebox General Toolset Extension.
The RecreatePassLines tool can be used to estimate combine harvester swath pass lines from yield points. In some cases, much like the project that John and I collaborated on, pass-line/swath information is not always stored in the point data created during harvesting or that information is not even captured and sometimes lost. Pass-line information is useful information when trying to re-construct, analyze and/or clean yield data. Our YieldFilter and YieldMap tool require information about the swath path for each point in the dataset. This tool can be thought of a pre-processing operation before running either of the more advanced yield mapping tools. The image below shows an example output from the RecreatePassLines tool, which is a network of vector lines (i.e. polylines) that have a “pass number” associated with each line in the attribute table data.
The YieldMap tool is the last tool currently part of the Precision Agriculture Toolbox. The YieldMap tool can be used to create a segmented-vector polygon yield map from a set of harvested crop points. The motivation from this tool comes from the idea that, the yield “point” itself is not a true representation of yield at the site of the point, but rather for an area behind that yield point that represents yield. This tool requires an accurate information about the swath pass lines and this user may use the RecreatePassLines tool to estimate swath line structures within the yield points. The image below is the output of the YieldMap tool, in this image yield has not been cleaned prior to executing this tool. However, it is recommended to filter/clean yield prior to using the YieldMap tool and an example of a clean YieldMap can be see down below as well.
This is an example of applying both the YieldFilter and YieldMap tool. Slide to slider to see the impact of cleaning/filtering yield using the YieldFilter tool on a yield map created from the YieldMap tool.
The Precision Agriculture Toolbox is something that John and I have been extremely excited to release and to continue development on. We believe that the tools in this toolbox and within our General Toolset Extension solve everyday problems that GIS professional encounter. You can download the Agriculture Extension for WhiteboxTools here. Be sure to let Whitebox Geospatial know when you do have an interesting application. If you are on twitter use the hashtag #WhiteboxTools or tweet us @whiteboxgeo; we love hearing about what you’re doing with WhiteboxTools.