Hochfrequenz/ebdamame
Python library to scrape .docx files with "Entscheidungsbaumdiagramm" tables into a truely machine readable structure
ebdamame
🇩🇪 Dieses Repository enthält ein Python-Paket namens ebdamame (früher: ebddocx2table), das genutzt werden kann, um aus .docx-Dateien maschinenlesbare Tabellen, die einen Entscheidungsbaum (EBD) modellieren, zu extrahieren (scrapen).
Diese Entscheidungsbäume sind Teil eines regulatorischen Regelwerks für die deutsche Energiewirtschaft und kommen in der Eingangsprüfung der Marktkommunikation zum Einsatz.
Die mit diesem Paket erstellten maschinenlesbaren Tabellen können mit rebdhuhn (früher: ebdtable2graph) in echte Graphen und Diagramme umgewandelt werden.
Exemplarische Ergebnisse des Scrapings finden sich als .json-Dateien im Repository machine-readable_entscheidungsbaumdiagramme.
🇬🇧 This repository contains the source code of the Python package ebdamame (formerly published as ebddocx2table).
Rationale
Assume that you want to analyse or visualize the Entscheidungsbaumdiagramme (EBD) by EDI@Energy.
The website edi-energy.de, as always, only provides you with PDF or Word files instead of really digitized data.
The package ebdamame scrapes the .docx files and returns data in a model defined in the "sister" package rebdhuhn (formerly known as ebdtable2graph).
Once you scraped the data (using this package) you can plot it with rebdhuhn.
Both packages together form the ebd_toolchain which scrapes EBD.docx files from the edi_energy_mirror and pushes them to machine_readable-entscheidungsbaumdiagramme.
How to use the package
In any case, install the repo from PyPI:
pip install ebdamameUse as a library
import json
from pathlib import Path
from ebdamame import get_ebd_docx_tables
from ebdamame.docxtableconverter import DocxTableConverter
docx_file_path = Path("unittests/test_data/ebd20230629_v34.docx")
# download this .docx File from edi-energy.de or find it in the unittests of this repository.
# https://github.com/Hochfrequenz/ebddocx2table/blob/main/unittests/test_data/ebd20230629_v34.docx
docx_tables = get_ebd_docx_tables(docx_file_path, ebd_key="E_0003")
converter = DocxTableConverter(
docx_tables,
ebd_key="E_0003",
ebd_name="E_0003_Bestellung der Aggregationsebene RZ prüfen",
chapter="MaBiS",
section="7.42.1"
)
result = converter.convert_docx_tables_to_ebd_table()
with open(Path("E_0003.json"), "w+", encoding="utf-8") as result_file:
# the result file can be found here:
# https://github.com/Hochfrequenz/machine-readable_entscheidungsbaumdiagramme/tree/main/FV2310
json.dump(result.model_dump(), result_file, ensure_ascii=False, indent=2, sort_keys=True)Use as a CLI tool
to be written
How to use this Repository on Your Machine (for development)
Please follow the instructions in our
Python Template Repository.
And for further information, see the Tox Repository.
Contribute
You are very welcome to contribute to this template repository by opening a pull request against the main branch.
Related Tools and Context
This repository is part of the Hochfrequenz Libraries and Tools for a truly digitized market communication.