Examples of object type customization

Binder

We believe all pydov types to have the most relevant fields available by default. However, more data might be available and of interest to your use case.

The pydov library provides a flexible way to customize the output of your data searches. One of the key features is the ability to customize the object types used in the search process. This notebook will demonstrate the different ways you can customize object types in pydov.

Defining Custom Object Types in pydov

The pydov library provides a flexible way to customize the output of your data searches. One of the key features is the ability to define your own custom object types, which allows you to include specific XML fields that are not part of the default object types.

This notebook will demonstrate three different ways you can define custom object types in pydov:

  1. Adding an XML field to an existing main type

  2. Adding an XML field to an existing subtype

  3. Creating a new subtype

Adding an XML field to a main type

To add an extra XML field to an existing main type, you can use the with_extra_fields method. This allows you to create a custom object type that includes additional fields beyond the default set.

[1]:

from pydov.search.boring import BoringSearch
from pydov.types.boring import Boring
from pydov.types.fields import XmlField

# Create a custom Boring type with an extra 'methode_xy' field
MyBoring = Boring.with_extra_fields([
    XmlField(name='methode_xy',
             source_xpath='/boring/ligging/metadata_locatiebepaling/methode',
             datatype='string')
])

# Search using the custom Boring type
bs = BoringSearch(objecttype=MyBoring)
df = bs.search(max_features=10)
df.head()

[000/001] .
[000/010] cccccccccc

[1]:
pkey_boring boornummer x y mv_mtaw start_boring_mtaw gemeente diepte_boring_van diepte_boring_tot datum_aanvang uitvoerder boorgatmeting methode_xy diepte_methode_van diepte_methode_tot boormethode
0 https://www.dov.vlaanderen.be/data/boring/2016... kb29d84e-B574 92424.0 170752.0 40.00 40.00 Wortegem-Petegem 0.0 0.0 NaN onbekend False gedigitaliseerd op topokaart 0.0 0.0 onbekend
1 https://www.dov.vlaanderen.be/data/boring/2016... kb29d84e-B575 92296.0 170885.0 46.00 46.00 Wortegem-Petegem 0.0 0.0 NaN onbekend False gedigitaliseerd op topokaart 0.0 0.0 onbekend
2 https://www.dov.vlaanderen.be/data/boring/1945... kb22d55e-B1037 108890.0 188690.0 14.31 14.31 Melle 0.0 15.0 1945-01-01 De Meyer L.-Gent False gedigitaliseerd op topokaart 0.0 15.0 droge boring
3 https://www.dov.vlaanderen.be/data/boring/2016... kb29d84e-B576 92243.0 170939.0 47.50 47.50 Wortegem-Petegem 0.0 0.0 NaN onbekend False gedigitaliseerd op topokaart 0.0 0.0 onbekend
4 https://www.dov.vlaanderen.be/data/boring/2016... kb29d84e-B577 92475.0 171088.0 39.00 39.00 Wortegem-Petegem 0.0 0.0 NaN onbekend False gedigitaliseerd op topokaart 0.0 0.0 onbekend

In this example, we’ve created a custom MyBoring type that includes an extra methode_xy field, which is retrieved from the XML document. We can then use this custom type in our search to include the additional field in the output dataframe.

Adding an XML field to a subtype

Similar to adding fields to a main type, you can also add extra XML fields to an existing subtype. This can be useful if you need to include additional data that is related to an existing subtype.

[2]:

from pydov.search.grondwaterfilter import GrondwaterFilterSearch
from pydov.types.grondwaterfilter import GrondwaterFilter, Peilmeting
from pydov.types.fields import XmlField
from pydov.util.location import WithinDistance, Point

# Create a custom Peilmeting subtype with an extra 'opmeter' field
MyPeilmeting = Peilmeting.with_extra_fields([
    XmlField(name='opmeter',
             source_xpath='/opmeter/naam',
             datatype='string')
])

# Create a custom GrondwaterFilter type with the custom Peilmeting subtype
MyGrondwaterFilter = GrondwaterFilter.with_subtype(MyPeilmeting)

# Search using the custom GrondwaterFilter type
fs = GrondwaterFilterSearch(objecttype=MyGrondwaterFilter)
df = fs.search(WithinDistance(Point(223063.20, 172028.31, epsg=31370), 10))
df.head()

[000/001] .
[000/003] ccc

[2]:
pkey_filter pkey_grondwaterlocatie gw_id filternummer filtertype x y start_grondwaterlocatie_mtaw mv_mtaw gemeente ... diepte_onderkant_filter lengte_filter datum tijdstip peil_mtaw betrouwbaarheid methode filterstatus filtertoestand opmeter
0 https://www.dov.vlaanderen.be/data/filter/2003... https://www.dov.vlaanderen.be/data/put/2017-00... 611/72/11 2 peilfilter 223063.2 172028.31 56.82 56.82 Kortessem ... 6.55 0.5 2004-04-06 NaN 55.02 goed peillint in rust 1 Labo
1 https://www.dov.vlaanderen.be/data/filter/2003... https://www.dov.vlaanderen.be/data/put/2017-00... 611/72/11 2 peilfilter 223063.2 172028.31 56.82 56.82 Kortessem ... 6.55 0.5 2004-05-19 NaN 54.45 goed peillint in rust 1 VMM - Afdeling Operationeel Waterbeheer
2 https://www.dov.vlaanderen.be/data/filter/2003... https://www.dov.vlaanderen.be/data/put/2017-00... 611/72/11 2 peilfilter 223063.2 172028.31 56.82 56.82 Kortessem ... 6.55 0.5 2004-12-14 NaN 55.11 goed peillint in rust 1 Labo
3 https://www.dov.vlaanderen.be/data/filter/2003... https://www.dov.vlaanderen.be/data/put/2017-00... 611/72/11 2 peilfilter 223063.2 172028.31 56.82 56.82 Kortessem ... 6.55 0.5 2005-02-24 NaN 55.97 goed peillint in rust 1 Labo
4 https://www.dov.vlaanderen.be/data/filter/2003... https://www.dov.vlaanderen.be/data/put/2017-00... 611/72/11 2 peilfilter 223063.2 172028.31 56.82 56.82 Kortessem ... 6.55 0.5 2005-09-29 NaN 53.46 goed peillint in rust 1 Labo

5 rows × 24 columns

In this example, we’ve created a custom MyPeilmeting subtype that includes an extra opmeter field. We then used this custom subtype to create a new MyGrondwaterFilter main type, which we can use in our search.

Creating a new subtype

In addition to adding fields to existing object types, you can also define your own custom subtypes from scratch. This allows you to include specific XML fields that are not part of the default object subtypes.

[3]:

from pydov.search.sondering import SonderingSearch
from pydov.types.sondering import Sondering
from pydov.types.abstract import AbstractDovSubType
from pydov.types.fields import XmlField

# Define a custom Technieken subtype
class Technieken(AbstractDovSubType):
    rootpath = './/sondering/sondeonderzoek/penetratietest/technieken'
    fields = [
        XmlField(name='techniek_diepte',
                 source_xpath='/diepte_techniek',
                 datatype='float'),
        XmlField(name='techniek',
                 source_xpath='/techniek',
                 datatype='string'),
        XmlField(name='techniek_andere',
                 source_xpath='/techniek_andere',
                 datatype='string')
    ]

# Create a custom Sondering type with the Technieken subtype
MySondering = Sondering.with_subtype(Technieken)

# Search using the custom Sondering type
ms = SonderingSearch(objecttype=MySondering)
df = ms.search(max_features=10)
df.head()

[000/001] .
[000/010] cccccccccc

[3]:
pkey_sondering sondeernummer x y mv_mtaw start_sondering_mtaw diepte_sondering_van diepte_sondering_tot datum_aanvang uitvoerder sondeermethode apparaat datum_gw_meting diepte_gw_m techniek_diepte techniek techniek_andere
0 https://www.dov.vlaanderen.be/data/sondering/2... VLA08-3.2-S16 218343.8 211622.9 51.15 51.15 0.0 12.80 2009-02-17 Labo Devlieger - Van Vooren discontinu mechanisch 200 kN NaT NaN NaN NaN NaN
1 https://www.dov.vlaanderen.be/data/sondering/1... GEO-92/114-S4 105658.0 188808.0 NaN 12.45 0.0 2.70 1992-10-15 MVG - Afdeling Geotechniek discontinu mechanisch 25 kN NaT NaN NaN NaN NaN
2 https://www.dov.vlaanderen.be/data/sondering/2... GEO-01/169-S23 189313.7 203475.9 NaN 21.40 1.2 15.15 2002-03-06 MVG - Afdeling Geotechniek continu elektrisch 200 kN - MAN2 2002-03-06 14:45:00 1.25 1.2 V NaN
3 https://www.dov.vlaanderen.be/data/sondering/2... GEO-01/169-S16 189633.5 203331.5 NaN 21.22 1.4 30.00 2002-03-11 MVG - Afdeling Geotechniek discontinu mechanisch 200 kN - MAN2 2002-03-11 13:50:00 1.22 9.2 V NaN
4 https://www.dov.vlaanderen.be/data/sondering/2... GEO-01/169-S16 189633.5 203331.5 NaN 21.22 1.4 30.00 2002-03-11 MVG - Afdeling Geotechniek discontinu mechanisch 200 kN - MAN2 2002-03-11 13:50:00 1.22 9.2 V NaN

In this example, we’ve defined a custom Technieken subtype that includes three new XML fields: techniek_diepte, techniek, and techniek_andere. These fields will be retrieved from the XML document and included in the output dataframe.

We then used the with_subtype method to create a custom MySondering object type that includes the Technieken subtype. This allows us to search for sonderingen (cone penetration tests) and retrieve the additional subtype fields in the output.

By defining custom subtypes, you can tailor the output dataframe to include the specific data that is most relevant for your use case. This provides a high degree of flexibility and control over the data you retrieve from the pydov data sources.