How data analysis can help identify marginal farmland


The power of data analysis in improving decision-making about future land use is highlighted by a project on the AHDB’s Strategic Farm East to use data to identify marginal land and land best suited for environmental programs.

The project, led by David Clarke, Soil and Agriculture Technician at Niab, has used 10 years of yield data along with operating and operating costs to create margin maps across 36 fields at EJ Barker and Sons’ Lodge Farm near Stowmarket in Suffolk to create.

See also: Tips for clamping sugar beets for optimal harvest storage

“That corresponds to more than 1.4 million data points, which is more than an Excel spreadsheet can hold. The challenge is to make sense of it and implement management decisions. “

To do this, he created a five-stage process as part of his doctoral project.

1. Clean yield data

There will always be inconsistencies in yield data, whether it’s because the header isn’t cutting at full width, the combine speed is changing, or it’s turning on the headland, says Clarke.

“If these points are not removed from combine harvester yield monitoring or any other analysis software, this can lead to quite drastic inconsistencies in the subsequent analysis, especially on the headland where there is a lot of turning.”

To combat this, he has used and developed the following methods of removing generated data:

  • Very slow or high combine speed
  • Where combine has quickly changed direction, pointing before and after
  • If data is very different from neighboring data (helps to remove cuts with reduced head width)

The yields can also be calibrated with in-house measured weighing platform or load cell data and corrected accordingly.

The adjusted yield maps typically have quite a few missed data points on the headland, he notes. “But enough precise data should be left behind from the headland, as in this case the Barkers drive around the headland two or three times while they open the field.”

2. Apply an economical price to each data point

While adjusted earnings data can begin to identify areas of high or poor performance, it is difficult to relate that directly to economic performance. “Our next step was to give each point an actual margin.

“Brian Barker has very accurate records for each of his fields so we can calculate either a gross or a net margin for each yield map point by multiplying the yield by the price of the grain minus the total cost to get the net margin.”

In Mr. Barker’s case, this was relatively easy as the entries in all fields were standard. It would be more complicated, however, if seeds with variable dosages, fertilizers or pesticides were used for more precise cultivation.

© Tim Scrivener

3. Identify areas of the field that are performing the same every year

Once the net margins are calculated, a statistical method called clustering can be used to identify areas of fields that are performing the same every year, he explains.

It compares every point in the field over the years to identify those that fluctuate in a similar pattern over time – for example, those that are always high yielding or low.

These can then be used to create a cluster map of the field highlighting similarly performing areas each year.

“You can then start to see what that means, for example by comparing wheat yields and margins for each area over the years.”

In the shrub field example (see graphic), five areas of different performance were found with this method, with the green and blue area (clusters 3 and 4) in the middle of the field consistently being the highest edge area of ​​the field with the orange area (cluster 1) next to is in a wooded area, usually the poorest part of the field, he says.

“If you look at the average margin across the rotation, the blue and green areas earn an average of £ 780-840 / ha. However, the orange cluster averaged just £ 240 ha between 2011 and 2015. “

Most of this area was taken out of production in 2019 and converted into 0.9 ha of pollen and nectar mix for £ 451 / ha – based on this analysis, the analysis has provided an economic justification for discontinuing this area from production.

In this case, while the data was used to justify a decision already made, the principle applies to future decisions, he says.

“We can identify other poorly performing clusters across the farm and then compare the costs of running a program, including setup and management costs and the total payment of the program, to make decisions based on the economic returns from arable production in the Compared to the ecological yields based options, ”he explains.

Result of the cluster analysis of the Shrubbery Field by David Clarke

Bush graphics

© Niab

Net margin (£ / ha) for each cluster in Shrubbery Field

Year (harvest)

Cluster 1

Cluster 2

Cluster 3

Cluster 4

Cluster 5

2011 (WW)






2012 (WW)






2013 (WOSR)






2014 (WW)






2015 (WW)






2019 (WW)






2020 (WW)












4. Use other data sources to further aid decision-making

Other data sources can also be helpful in making decisions, especially about future land use and the entry into environmental responsibility. “For example, mapping the risk of erosion using topographical wetness and lidar elevation data can help identify areas that are potentially at risk from ground or water movement.

“That way you can look for areas that are at high environmental risk and underperforming and where they coexist.”

5. Compare between fields and in the field

The analysis also allows you to compare fields across the farm, especially if fields were in the same crop rotation, says Clarke.

Individual field analysis allows the underperforming areas to be identified, but it is unlikely that parts of each field will be taken out of production. Cross-field analyzes offer the possibility of highlighting the underperforming parts of the operation and giving better insight into where action should be taken, he explains.

What data do you need?

  • Yield mapping (calibrated) – essential
  • Variable costs at field / plant level – essential
  • Recorded field yield values ​​- useful
  • Non-spatially assigned harvest information – useful
  • Ground scans – bonus
  • Satellite imagery – bonus
  • Environmental Risk Cards – Bonus

What tools are available for this type of analysis?

While David Clarke’s analysis is tailored to the project and uses advanced data cleansing and clusters, there are some digital tools that could potentially be used to similarly identify marginal land.

Farmplan’s gatekeeper can accurately reflect Mr. Clarke’s analysis. Yield maps can be loaded into the platform via API cloud connections for most major combine brands or transferred via USB. Anomalies can be eliminated by setting up a template that can be run year after year and corrected using scale tickets, explains Adam Joslin, Training Services Manager at Farmplan.

The platform’s crop management information can then be matched with the yield maps to produce either net or gross margin maps on a field basis and accommodate all variable rate applications.

But while yield maps can be aggregated over several years, the edge maps cannot yet, notes Mr. Joslin.

Hutchinson’s Omnia can also generate production cost maps, including operating costs, which are matched against yield maps, while Frontier’s MySoyl maps total yield, gross and net margin after entering fixed, variable and sales values ​​for the harvest for both a single and multi-year period can.

David Clarke spoke about borderland identification during two AHDB webinars


Comments are closed.