Release notes#

Version 0.27.0#

Enhancements

  • Bugfix for contribution analysis.

  • Disaggregated contributions from direct elementary flows.

  • Bugfix for market construction.

  • Bugfix for transport and trade margins.

  • Added new tasks for by-product replacing.

  • Added new task for SimaPro-compliant format.

  • Bugfix for zero-footprint in by-product market.

  • Modified approach for household disaggregated footprint.

  • Added method in Matrix to detect circular dependency error.

  • Fixed PathBuilder.prices.

  • Adjusted by-technology modelling approach from market products to industrial products.

  • Fixed mass balance for fertilizer market.

  • Updated prices for FAOSTAT data.

  • Fixed an issue with technical coefficients for sorting activities.

  • Improved documentation of methodology.

Backward Incompatible Changes

  • Removed unused utilities.

  • Removed unused tasks related to market.

  • Deprecated paths.hiot_with_iluc.

Version 0.26.2#

Enhancements

  • Released V2.0.2.

  • Migrated a component of the household-disaggregation pipeline from R to Python.

  • Introduced an aggregated “average household” profile for all countries/regions where detailed household-level disaggregation is available.

  • Bugfix related to intensification of cropland.

  • Bugfix related to waste treatment services.

  • Bugfix related to fertilizer market.

  • Better technological modelling with more technical coefficients.

  • Updated documentation.

Backward Incompatible Changes

  • Deprecated bugfixing task: revise_coproduction_of_pulp_industry, revise_unit_of_waste_services, determine_direct_emissions, scale_up_combustion_activities.

  • Removed ununsed utilities.

Version 0.26.1#

Enhancements

  • Fixed schema change for final demand footprint.

  • Removed partial aggregation in contribution analysis.

  • Other updates for database connection.

Backward Incompatible Changes

  • Cleaned up configuration parameters.

  • Deprecated unused tasks.

Version 0.26.0#

Enhancements

  • Fixed final demand account in generate_fd_b2.

  • Replaced flow_type from activity classification from product to industry.

  • Adjusted create_average_market for flexible monetary market composition.

  • Improved Connector.put for product correspondence.

  • Changes for flexible market and descriptions.

  • Bugfix for nuclear fuel trade.

  • Added transfer coefficient constraints in balance.

  • Added validation against Ecoinvent data.

  • Adjust technical coefficient constraints in balance.

  • Improved documentation for balance.

Backward Incompatible Changes

  • Deprecated generate_hiot_for_simapro.

  • Other deprecations for unused tasks.

Version 0.25.1#

Enhancements

  • Fixed documentation format for glossary.

  • Expanded methodology documentation for sector modelling, balances, and consequential workflows.

  • Expanded methodology overview with inline table of contents and references.

  • Rewrote documentation getting-started guide for environment setup.

  • Improved Node validation.

  • Added SUTGraph.prune to prune invalid nodes.

  • Added on-page feedback buttons to collect quick GitLab reactions.

  • Added footprint validation before loading into database.

Backward Incompatible Changes

  • Removed un-used functions in capital endogenization.

  • Removed un-used tests.

Version 0.25.0#

Enhancements

  • Added parametric constraints in balancing.

  • Bug fix for the task create_average_markets.

  • Improved code readability.

  • Added description field to Node.

  • Improved monetary SUT workflow.

  • Added agent instructions.

  • Improved functionality of Connector.post.

  • Security update.

Backward Incompatible Changes

  • Removed un-used files in documentation.

Version 0.24.4#

Enhancements

  • Added a single task for building domestic mix.

  • Refactored load PPF tasks for batch ingestion.

  • Integrated gapfilling into the workflow.

  • Added trade of services.

  • Improved documentation of PPF integration.

Backward Incompatible Changes

  • Deprecated domestic mix for each product.

Version 0.24.3#

Enhancements

  • Fixed integration of coke in the waste account.

  • Renamed asses_waste_from_stock to calc_waste_stock_variation.

  • Added all accounts for monetary supply and use tables.

  • Fixed task dependencies for MSUT-related tasks.

  • Minor fix for PPF integration documentation.

  • Added load_ppf_emission as a task.

  • Renamed coke related tasks to ppf tasks.

Backward Incompatible Changes

  • Deprecated a few utility functions.

  • Depreacted a few un-used functions in the agriculture module.

Version 0.24.2#

Enhancements

  • Fixed national mix use of coke.

  • Used emission account after balance for footprint.

  • Excluded EXIOBASE monetary trade.

  • Fixed correspondence for cattle meat from Concito.

  • Fixed aluminum recycling by-product.

Backward Incompatible Changes

  • Deprecated task distribute_remaining_use_with_monet_keys.

  • Bugfix for integrating supply of coke PPF.

  • Deprecated un-used utilities.

Version 0.24.1#

Enhancements

  • Fixed monetary use distribution key.

  • Fixed location mapping from EUROSTAT to BONSAI.

  • Refactored integration of PPF supply.

  • Other bugfix and readability update.

Version 0.24.0#

Enhancements

  • Added internal Monetary SUT.

  • Improved workflow dependency.

  • Separated Leontif inverse and footprint calculation.

  • Fixed task for footprint in monetary unit.

  • Other bugfixes and code structure optimization.

Backward Incompatible Changes

  • Deprecated footprint-related tasks.

Version 0.23.4#

Enhancements

  • Added production and final demand footprint for GTP20.

  • Refactored tasks related to FAOSTAT to align with LCI structure.

  • Fixed tasks related to ILUC.

  • Other documentation and code readability update.

Backward Incompatible Changes

  • Deprecated footprint-related tasks.

Version 0.23.3#

Enhancements

  • Added topological sorting for Workflow to avoid duplicated execution of tasks.

  • Fixed task dependency issue.

  • Fixed ryp2 optional package dependency issue.

  • Refactored contribution analysis for product-by-product IOT.

  • Fixed workflow not stop when task fail.

  • Other documentation update

Backward Incompatible Changes

  • Deprecated Task.is_ready().

  • Deprecated Task.visual_task_dependency().

Version 0.23.2#

Enhancements

  • Improved algorithm for by-production technology modelling.

  • Improved algorithm for reference product identification.

  • Integrated household disaggregation into workflow.

  • Improved use table validation.

  • Other bugfixes and documentation.

Backward Incompatible Changes

  • Deprecated BasePaths class in favor of a single PathBuilder.

  • Deprecated generate_prices_for_hiot.

Version 0.23.1#

Enhancements

  • Added price constraints to the balance.

  • Used dataio.resources.ResourceRepository instead of dataio.config.Config for integration.

  • Refactored energy SUT for better clarification between clean and load.

  • Bugfixes for coal-related activities.

  • Updated correspondence from EXIO3 code.

  • Other readability update.

Backward Incompatible Changes

  • Deprecated merge_emissions_account in favor of merge_all_emissions.

  • Removed DEBUG configuration.

  • Removed clean_loc attribute of Config.

Version 0.23.0#

Enhancements

  • New UI for documentation.

  • Refactored the algorithm for final demand footprint/contribution to calculate per capita footprint/contribution.

  • Integrated the first Parametrized production function.

  • Other documentation and code update.

Version 0.22.1#

Enhancements

  • Enhanced the inverse algorithm for cupy-based inversion.

  • Bugfix for reference/by- product list.

  • Switched from numpy to scipy.sparse for the default matrix inverse algorithm.

  • Tested compatibility of critical tasks with Python3.12.

  • Refactored generate_fd_simapro task and renamed it to generate_fd_b2.

Version 0.22.0#

Enhancements

  • Refactored the algorithm for creating the Stone’s market.

  • Refactored the algorithm for squarizing the supply and use table.

  • Enhanced the algorithm for calculating footprint.

  • Enahced the algorithm for creating the b2 Version for HIOT.

Backward Incompatible Changes

  • Adjusted workflow and deprecated few tasks.

Version 0.21.0#

Enhancements

  • Refactored the routine for FAOSTAT data processing

  • Converted BigClimate data into LCI data format

  • Added USGS data cleaning

  • Enhanced algorithm for balancing.

  • Bugfix for Stone’s market.

  • Added classifications package dependency.

  • Documentation update.

Version 0.20.0#

Enhancements

  • Fixed sourcing of new version file when not available.

  • Refactored to use reference product for constraints in co-production.

  • Added sut_resource_repository to Config.

  • Added task to extract energy efficiencies.

  • Enhanced the balance algorithm.

  • Fixed property update for nodes in graph.

  • Added rdflib as a dependency.

  • Added RDF graph implementation to SUTGraph.

  • Refactored task create_stone_market.

Backward Incompatible Changes

  • Deprecated task_status in Task.run method.

Version 0.19.0#

Enhancements

  • Compatibility update for the Workflow and Task class.

  • Bugfix for allocate_FAO_feed and add_by_products_to_market_feed.

  • Disaggregated several activities to fix by-production.

  • Removed unused scripts.

  • Improved documentation on integration of parametrized production function.

  • Added notebook to do country emission comparison.

Version 0.18.6#

Enhancements

  • Enhanced balance algorithm.

  • Added task dependency and its network visualization function in task class.

  • Documentation update.

Version 0.18.5#

Enhancements

  • Enhanced balance algorithm.

  • Upgraded ipopt solver from 3.11 to 3.14.

  • Bugfix for task registration.

  • Documentation update.

Version 0.18.4#

Enhancements

  • Fixed clean_fao_products_transfer_product_coeff.

  • Associated versions with configuration date.

  • Moved version directory to BONSAI_HOME.

  • Improved documentation on integration of parametrized production function.

  • Refactored tasks reltaed to the emission account.

  • Added calculate_output_proportionality_coeff task to estimate by-production proportion.

  • Other documentation update.

Version 0.18.3#

Enhancements

  • Excluded C_MOTO and C_OTRE from balance.

  • Fixed cyclic task dependency among prices, hybrid SUT, and transport. #365

  • Fixed correspondence issue with distribute_agri_products_used_as_seed. #364

  • Fixed an issue with replace_activities_with_markets.

  • Added deprecation warnings for some utility functions.

  • Other code and documentation update.

Backward Incompatible Changes

  • Deprecated creates_initial_use_accounts.

  • Removed exiogen_data_for_distrib_coeffs.py.

Version 0.18.2#

Enhancements

  • Bugfixs for tasks.

  • Removed global market in SUTs.

  • Added tonnes (service) as a property layer for waste treatment service.

  • Updated documentation on methodology and PPF integration

  • Bugfix for feed use estimate.

  • Adjusted gap-filling procedure for price.

  • Added waste stock estimate.

Version 0.18.1#

Enhancements

  • Updated build_hybrid_sut to new workflow and other simplifications.

  • Fixed an issue in monetary SUT build.

  • Updated Task.info and overide Task.__repr__ to use Task.info.

  • Other bugfix and code readability improvement.

  • Updated documentation.

Version 0.18.0#

Enhancements

  • Added task dependencies for all tasks.

  • Upgraded dependency on the transport module to use Transport.distance.

  • Used https connection to fetch the transport package.

  • Finalized the market account and created the first draft for supply and use table before balancing.

  • Added new algorithms to gapfill prices.

  • Added a version folder for the version file.

  • Refactored algorithms related to trade for readability.

Version 0.17.3#

Enhancements

  • Fixed the aggregate_faostat_accounts task.

  • Updated task sequence related to the B2 version.

  • Added to_database attribute to the Config class.

  • Fixed a correspondence error in estimate_ferts_supply_n_property.

  • Fixed a schema error in calculate_ferts_markets.

  • Added automatic identification of task_name with .get_dataframe_with_task in CSVResourceRepository.

  • Added the collect_fao_exchange_rate task.

  • Fixed path error for clean_exio_3.

Backward Incompatible Changes

  • Deprecated log_level and log_handler instance attributes of Config.

Version 0.17.2#

Enhancements

  • Added list_engines method for Matrix.

  • Added dependency for scikit-learn.

  • Fixed error in clean_land_use.

  • Bugfix and other documentation updates.

Backward Incompatible Changes

  • Deprecated ut.report_task in favor of Task.log.

  • Deprecated auto_api for documentation.

Version 0.17.1#

Enhancements

  • Added parallel argument to allow parallelization of route searching.

  • Added dependency to rpy2 for R scripts integration.

  • Fixed an issue in freight market estimation.

  • Modified the method to derive freight rate and inventory cost.

Version 0.17.0#

Enhancements

  • Fixed wrong mapping in price estimation of market products.

  • Added Route to interact with transport package.

  • Added FreightTranport with logit Choice model to estimate transport modal share.

  • Renamed calc_freight_transport_use to calc_freight_use.

  • Added merge_freight_use.

  • Added package use and trade estimate.

  • Refactored market account estimation.

Version 0.16.2#

Enhancements

  • Fixed token setting with paginated results.

  • Uploaded forestry product correspondence to API.

  • Added passenger transport from OECD.

  • Added gapfill tasks for freight transport.

  • Cleaned UN production data.

Version 0.16.1#

Enhancements

  • Removed task related fuel use by transport due to duplication.

  • Added freight transport API fetch from OECD.

  • Estimated freight and passenger transport supply.

  • Deprecated few PathBuilder instance attributes.

  • Added estimate of package supply.

  • Added estimate of treatment activity.

  • Other bug fixes.

Version 0.16.0#

Enhancements

  • Added dependency on the transport package.

  • Fixed token fetch error in DataReader.

  • Modified a validation check in mapping.

  • Added the transport account.

  • Fixed API request error with no-token access.

Version 0.15.1#

Enhancements

  • Added package and residual agricultural product use.

  • Added package waste supply and use.

  • Added example code for classification disaggregation.

  • Updated documentation.

Version 0.15.0#

Enhancements

  • Added token_use configuration to avoid duplicated token-use prompt. #322

  • Moved token setting related methods from Connector to ConnectorFactory.

  • Created a SingletonMeta class for Config to avoid duplicated initiation of configuration.

  • Removed rough code from src/.

Version 0.14.3#

Enhancements

  • Splitted monetary SUT data into distinct accounts.

  • Improved use of classifications.

  • Removed unused task related to monetary distribution coefficients.

  • Removed trade_route module.

  • Added transport module.

Version 0.14.2#

Enhancements

  • Fixed bugs related to PathBuilder.

  • Added tasks to estimate package use and package waste.

  • Added task to aggregate BONSAI back to EXIOBASE.

Backward Incompatible Changes

  • Deprecated few path attributes of PathBuilder.

Version 0.14.1#

Enhancements

  • Automated env var BONSAI_TOKEN setting through user input. #318

  • Added token caching.

  • Added token creating API endpoint into APIEndpoint enum.

  • Deleted deps on frictionless.

  • Fixed Task.run by giving default parameters.

  • Fixed several path instances.

Version 0.14.0#

Enhancements

  • Separated import_MSUTs_n_generates_distr_coeffs into clean_monetary_supply and clean_monetary_use_n_fd.

  • Modified the behavior of PathBuilder.compose so that account_name is no longer mandatory.

  • Added a task to convert BONSAI classification to EXIOBASE classification.

  • Added OS platform specific marker for pypardiso.

  • Added cupy as an optional engine for matrix inverse.

  • Added the Matrix class.

  • Set Python image as 3.11 in CI pipeline.

Version 0.13.1#

Enhancements

  • Replaced several correspondence files with correspondence API.

  • Added draft code to automatically disaggregate package classification.

Version 0.13.0#

Enhancements

  • Started to use APIs for classifications update.

  • Started to use APIs for correspondence mapping.

  • Created an example of interacting with APIs.

  • Added a new API endpoint for property values.

  • Fixed collect_trade_matrix with parquet file streaming.

  • Created a validate module to aggregate all the data validation utilities.

Backward Incompatible Changes

  • Deprecated DataPackage in favor of Resource.

Version 0.12.6#

Enhancements

  • Code readability update.

  • Documentation update.

Version 0.12.5#

Enhancements

  • Modified the schema of the product property account.

  • Replaced csv-based location correspondence with the location correspondence API.

  • Added put method for Connector to update records in APIs.

  • Replaced csv-based activity classification with the activity classification API.

  • Replaced csv-based product unit classifiction with the product classification API.

  • Updated documentation.

Backward Incompatible Changes

  • Deprecated exio_unit in favor of PropertyEnum.

  • Depreacted the use of unit_for_residual_products.xlsx.

Version 0.12.4#

Enhancements

  • Fixed errors in the estimation of global market supply and use.

  • Created a merged product property account.

  • Excluded the item count layer from balance.

  • Added energy content estimation for market products. #312

  • Added fertilizer LCI and nitrogen aborption estimation.

Version 0.12.3#

Enhancements

  • Fixed task dependencies of balance_sut.

  • Added ore, fuel, and extraction and CO2 absorption into the balancing model.

  • Added energy layer into the balancing model.

  • Rename property_of_products to merge_property_of_products.

  • Added resource heat values and calculated energy supplies of resources.

  • Added the squared difference method for objective function setting.

Backward Incompatible Changes

  • Deprecated IndexNames.RESOURCE_NAME in favor of IndexNames.RESOURCE.

Version 0.12.2#

Enhancements

  • Renamed build_sut_for_balancing to build_hybrid_sut.

  • Added convert_sut_to_graph as a seperate task.

  • Added build_monetary_sut as a task to create supply and use tables in monetary units.

  • Added price-derived monetary supply and use table in the balance model.

  • Added resource extensions to the balance. #302

  • Other code readability improvement.

Version 0.12.1#

Enhancements

  • Added emission flows into graph structure.

  • Added emission flows into balancing.

  • Fixed a duplication error in adding .inflows and .outflows to Node.

  • Added calc_prices_market_products and estimate_prices_other_products to fill missing price information.

Version 0.12.0#

Enhancements

  • Refactored the build_sut_for_balancing task.

  • Fixed the merge_emission_accounts task.

  • Improved the performance of validate_graph_balance method of SUTGraph.

  • Used graph as the data structure required by the balance model.

  • Used uuid for node indexing to avoid index collision.

  • Used PathBuilder.compose for balance files.

Version 0.11.5#

Enhancements

  • Fixed the create_one_supply task.

  • Fixed an unit conversion error in waste treatment price. #292

  • Fixed double units for M_eggsl and M_eggpl for the supply table. #293

  • Fixed an error in reading the correspondence of waste market to waste fraction.

  • Added C_CHAR in other_prod_v3_vs_v4.csv correspondence to fix a correspondence error in distrubuting product use.

  • Fixed a correspondence error that causes missing fertilizer supply from China. #299

Backward Incompatible Changes

  • Deprecated PathBuilder.baci_clean.

Version 0.11.4#

Enhancements

  • Added balance constraints for activities

  • Splitted out each layer for balance validation in the validate_node_balance method.

  • Renamed the determine_heat_markets_cons_elec task to separate_heat_n_elec_consumption.

  • Added task dependencies for the energy module and the LCI module.

  • Fixed the clean_IEA_data task.

Backward Incompatible Changes

  • Deprecated PathBuilder.iea_source_data.

Version 0.11.3#

Enhancements

  • Added methodology documentation for by-product modelling, marginal electricity mix and capital endogenization.

  • Added related tasks to methodology sections.

  • Removed duplicated tasks in task registry.

Version 0.11.2#

Enhancements

  • Introduced property layer in the balance problem formulation.

  • Added tasks for packaging coefficient clean, and packing waste calculation.

  • Renamed clean_eurostat_waste to clean_eurostat_waste_treatment.

  • Added tasks to disaggregate residential waste into industrial waste and municipality waste.

  • Added export_graph method to BalanceModel.

  • Added constraints for activity balance but not used because of not able to find an optimum.

Backward Incompatible Changes

  • Deprecated export_results method of BalanceModel.

Version 0.11.1#

Enhancements

  • Added validate_balance to the SUTGraph class for graph-level balance validation.

  • Renamed node-level balance from validate_balance to validate_balance_node.

  • Fixed missing supply in balancing procedure.

  • Refactored validate_balance method to validate balance on multiple properties layers.

Backward Incompatible Changes

  • Deprecated the validate_graph_balance utility function.

  • Deprecated the balance_node method in SUTGraph.

Version 0.11.0#

Enhancements

  • Added the first working balance model.

  • Added closest match suggestion for Workflow.get_task.

  • Finished product balance model and consolidated the workflow for balance.

  • Added a script to analysze balance results.

Version 0.10.5#

Enhancements

  • Added gitlab url for documentations.

  • Fixed documentation errors and warnings from sphinx.

  • Added contribution guideline for Documentation.

  • Added favicon for the docs.

  • Refactored product supply and use balance validation and optimization.

Version 0.10.4#

Enhancements

  • Added BONSAI color scheme for documentation website

Version 0.10.3#

Enhancements

  • Added documentation on land use accounting.

  • Added CI pipeline for automatic documentation creating. #278

  • Added __version__.py to fix version in documentation.

  • Switched to autoapi for automatic API documentation.

  • Fixed cross reference errors in readme.

Version 0.10.2#

Enhancements

  • Added .log method in the Task class.

  • Started to deprecate report_task function.

  • Removed the use of ut.report_task and ut.write_a_report.

  • Added task dependencies for packaging related tasks.

  • Improved docstring for PathBuilder.compose.

Version 0.10.1#

Enhancements

  • Modified the triggering conditions of tasks related to basic price footprint calculation.

  • Cleaned up hiot code related to footprint hybrid and footprint basic price.

  • Moved LCI clean data into clean directory and modified the paths.lci_cleaned attribute of Paths.

  • Moved account list into a persistent storage instead of in the repository to facilitate concurrent writing.

  • Added task dependency for food waste related tasks.

Version 0.10.0#

Enhancements

  • Cleaned up code related to HIOT manipulation.

  • Added deprecation warning to the exio_unit class.

  • Added generate_tidy_io into the workflow.

  • Miscenaneous code refactoring and documentation update.

  • Added Packaging account and tasks related to calculating package coefficients, and waste from packages.

  • Added human metabolism coefficient to calculate waste coefficient from final consumption.

  • Added a task determine_total_waste_of_products to aggregate waste supply account.

  • Added stock degradation function (waste lifetime).

  • Disaggregated waste into industrial waste and municipality waste.

Backward Incompatible Changes

  • Depreacted the create_compact_io task.

  • Renamed calc_hiot_n_footprint to calc_footprint_b1.

  • Renamed utility_exiobase_3.py to util_others.py.

Version 0.9.3#

Enhancements

  • Introduced pypardiso to solve A matrix inversion.

  • Full matrix inversion for b1, iLUC, marginal electricity and capital endogenization version using pypardiso.

  • Improved the algorithm for updating HIOT with marginal electricity mix data.

Version 0.9.2#

Enhancements

  • Created a unified table for transfer coefficients.

  • Created a unified table for waste coefficients.

  • Created a unified table for emission coefficients.

  • Fixed an unit issue in capital endogenization.

  • Bugfix with the import_lci_alcolic_beverage task.

  • Added byproduction of beverage into the Supply table.

  • Add byproduction of feed into the feed market.

  • Fixed an unit issue with the clean_fao_forestry task.

  • Added PropertyEnum to standardize the use of units across the repository.

  • Downgraded account not found infomation in VersionCollection from warning to info.

  • Added direct activation of tasks within the DEBUG mode.

  • Upgraded faostat api to v1.1.1 because the service URL has changed.

  • Added a first version of tidy IO implementation. (Not included in the workflow yet).

Backward Incompatible Changes

  • Moved default settings to the first level of support.

  • Renamed config_data.py as config.py. The Config class will be used for configuration loading in the future.

  • Replaced exio_unit with PropertyEnum.

Version 0.9.1#

Enhancements

  • Refactored waste price estimation and added into task registration.

  • Refactored electricity price estimation and added into task registration.

  • Added task dependencies for price related tasks.

  • Started to deprecate data packages.

  • Integrated BACI data collection into the workflow.

  • Other documentation updates.

Backward Incompatible Changes

  • Deprecated lci_monet_generic attribute for PathBuilder becuase it is same as lci_prod_generic.

  • Modified the default behavoir of the PathBuilder.compose. Now it defaults to get the latest version of an account (Before it is None). To set a no-version path, version_date has to be set as None explicitly.

Version 0.9.0 (Renaming to BONSAI-IO)#

Version 0.8.3#

Enhancements

  • Bugfix for the import_n_clean_IEA_data task. #289

  • Performance improvement for the import_n_clean_IEA_data task.

  • Bugfix for the import_n_convert_un_data task.

  • Consolidated workflow related to UN data.

  • Added a task clean_prices_recycling that derives prices for product recycling. #290

  • Added a task clean_elec_prices that derives electricity prices from ComTrade.

  • Added reference product to energy-related transfer coeffients. #291

  • Bugfix in the generate_market_of_heat task.

  • Bugfix for the clean_fao_forestry task.

  • Added a task calculate_by_production_of_beverage to calculate by-production of beverage and inserted into workflow.

Known issues

  • Missing activity correspondence between IEA and Base classifications causing error in the clean_IEA_data task.

Version 0.8.2#

Enhancements

  • Fixed a bug in testing zeros in the diagonal of hiot_b2_extend.pkl. #285

  • Converted supply use tables into a graph and do a node balance.

  • Fixed an issue in the import_n_clean_IEA_data task.

Version 0.8.1#

Enhancements

  • Fixed bugs related to unit and column name changes when integrating with FAOSTAT API with iLUC model.

  • Improved FAO clean task to accommodate multiple years.

  • Fixed a bug related to PathBuilder.compose. Account source and version source not synchromized when an account is first created.

  • Fixed bug for the calc_tier1_contrib task and improved performance.

  • Added the aggr_tier1_contrib task to aggregate top contributions and post to the database.

  • Added chunk_size to the post method of Connector for lightweighted post.

Version 0.8.0#

Enhancements

  • Improved memory usage for the generate_hiot_simapro task.

  • Fixed a bug in PathBuilder.compose.

  • Added is_downstream_task method for the Task class.

  • Added the stop arugment for Workflow.execute.

  • Fixed the collect_trade_matrix task for FAOSTAT.

Backward Incompatible Changes

  • Renamed the start_from argument of the Workflow.execute method to start.

  • Deprecated Config.exnames.

Version 0.7.2#

  • Added collect_comtrade_trade task. (Known issue: quota exceeding error)

  • Added concat_footprint and calc_tier1_contrib into the workflow.

  • Fixed a bug in PathBuilder.compose.

  • Refactored the contribution analysis for speed.

  • Improved ReadMe document.

Version 0.7.1#

  • Renamed determine_total_demand_of_manure to clean_concito_manure_total.

  • Added sorting for listed groups and tasks in Workflow().list_tasks().

  • Added clean_baci_price into the workflow.

  • Added collect_baci and collect_baci_unzip tasks. (Still not working with clean tasks)

  • Added scripts related to PV life cycle inventories.

  • Fixed bug in fertiliser market availability.

Version 0.7.0#

  • Fixed a bug in the combine_trade_flow task after integrating with faostat API.

  • Refactored distribution of monetary flows from exio3 into the use table.

  • Fixed a bug in merging LCI.

  • Moved calculation of available physical flow from final use table to inital use table.

  • Fixed a bug in generating FAOSTAT clean accounts that generate unintended nan values.

  • Refactored food distribution for the use table.

  • Refactored seed distribution for the use table.

  • Added extensions (Natural resource and value added) to the SimaPro version.

  • Refactored marginal electricity market insertion logic for SimaPro

  • Added Account and AccountRepository classes for accounts data access.

  • Added VersionRepository interface and TXTVersionRepository and APIVersionRepository class to manage version data access logic.

  • Refactored PathBuilder().create method to .compose so that it can compose a path with a given pattern -> /<stage>/<account>/<version_date>/

  • Added a test for Workflow init.

Version 0.6.0#

  • Fixed a bug in url parse in the Resource class.

  • Fixed a bug in the clean_fao_forestry task.

  • Added dependency to the faostat package.

  • Added colleact_faostat related tasks.

Version 0.5.5#

  • Renamed import_baci_for_exiobase to clean_baci_trade.

  • Added clean_baci_w_trade.py in the trade module to bridge the gap between collect and clean BACI data.

  • Added the aggregate_baci task into task graph.

  • Used schema_enums for the combine_trade_flows task.

  • Added waste and production related tasks into the task graph.

  • Added test for hiot market balance.

Version 0.5.4#

  • Upgraded dependecy to Pandas from 2.0.0 to 2.0.3 to resolve redundant warnings.

  • Renamed elaborate_faostat task to aggregate_faostat_accounts and added upstream task dependencies for it.

  • Renamed fao_foresty_data to clean_fao_forestry and added task dependencies for it.

  • Improved the speed of the clean_fao_fishery task.

  • Miscellaneous code refactor for readability improvement.

Version 0.5.3#

  • Fixed a location parse error in Resource.

  • Modularized the construction of use table for easier decomposition.

  • Added clean_AU_waste, clean_eurostat_waste and clean_waste_exio3 tasks.

Version 0.5.2#

  • Fixed a FileNotFound bug in task elaborate_faostat.

  • Fixed a bug in task land_use_extensions.

  • Fixed a query error bug in ResourceRepository.get_from_dataframe method.

  • Fixed an error in FAOSTAT country correspondence in task elaborate_faostat.

Version 0.5.1#

  • Added .upstream_tasks and .downstream_tasks method in the Task class to create task dependency.

  • Added TaskStatus to track the status of tasks in workflow.

  • Workflow().execute() works now.

  • Added tasks dependencies in task registration.

  • Added task .is_ready() check to check the status of a task.

  • BONSAI_HOME can now be set as /path/to/bonsai/_bonsai/ as well.

  • Updated ReadMe docs.

Version 0.5.0#

  • Added Resource and ResourceRepository classes for data resource management.

  • Added CSVResourceRepository and SQLiteResourceRepository for different implementations of a resource repository.

  • Added .resource_repository property for Config.

  • Deprecated ._set_bonsai_root method in PathBuilder. Now the environmental variable BONSAI_HOME is the only way to initiate the workflow.

  • Improved performance for the clean_sua_crop_lives task.

  • Fixed bugs in the trade_route module.

  • Updated docs on data access and quick examples.

Version 0.4.3#

  • Added distribute_IEA_final_consumption_to_activities task to allocate energy across final consumption activities.

  • Optimized sparse data representation to decrease the size of exported datasets.

  • Fixed mapping of IEA heat markets and base activities classification.

  • Fixed an issue in MSUT unit labelling, changing from “euro” to “Meuro”.

  • Miscellaneous bug fix and docs update.

Version 0.4.2#

  • Deprecated exnames in favor of schema_enums.

  • Register all tasks in workflow_main.py to task_registry.

  • Improved documentation for Task.

  • Added .run_tasks() method to run several tasks.

  • Renamed the workflow_main module to task_registration

  • Updated documentation on setting up environmental variable.

  • Wrapped up all tasks into task_registry.

Version 0.4.1#

  • Updated dependence to pandas==2.0 and related code refactoring.

  • Fixed bugs in clean FAOSTAT data.

  • Deprecated paths_exio property for Config.

  • Added deprecation warning to exnames property.

  • Added schema_enums property to replace exnames in the Config class.

  • Added format for Workflow.list_tasks() method for better usability.

  • Fixed a bug in ferlizer cleaning.

  • Added .get_task_info() method for the Workflow class.

Version 0.4.0#

  • Finished first version of the Workflow class and the Task class.

  • Created a new way to register tasks to workflow.

  • Added templates for contributing.

  • Other bug fixs and documentation update.

  • Bumped BONSAI version to 1.1.0

Version 0.3.3#

  • Fixed path error to b2_simapro_version.

  • Miscellaneous code clean up.

Version 0.3.2#

  • Inserted revised LCI data for fertilisers

  • Removed unexpected input of fertilisers into energy/electricity processes

  • Corrected null margins for outputs from multi-functional activities

  • Renamed the exio_names module as schema_enums and decoupled from tasks

  • Added a database level versioning variable DB_VERSION

Version 0.3.1#

  • Added .list_path_attributes method to PathBuilder.

  • Cleaned up utilities from exiobase3.

  • Updated documentation on installation and quick start in Readme.md.

  • Renames capital_invest_mat_rev2.py as capital_invest_mat.py.

  • Added a logging utility: deprecated decorator.

  • Added deprecation warning to Config.paths_exio, ready to replace .paths_exio with .path_repository.

  • Moved report_change_in_versions function to loggings.py for better logging management.

  • Created methodology.md and moved detailed methodology descriptions.

Version 0.3.0#

  • The ExioPaths class is renamed as PathBuilder for generalization of naming.

  • ._path_exio property of the Config class is renamed as _path.

  • version_file argument is deprecated in the PathManager class.

  • .list_parameters() method is added for the Config class for better exploration of the configuration by users.

  • Added task decorator to register tasks to a global register for workflow to find.

  • Added list_tasks and discover_tasks to the Workflow class.

  • Incorporated path creating logic into PathBuilder.

Version 0.2.1#

  • Added a task to aggregate footprint in different units.

Version 0.2.0#

This release introduces significant architectural improvements to the project, focusing on enhancing the workflow execution and configuration management. Key developments include the introduction of a Workflow class for streamlined workflow management and a Config class for centralized configuration handling.

  • Created Workflow class to to orchestrate the execution of tasks, making the workflow process more intuitive and manageable.

  • Restructured the project hierarchy, moving Config and Workflow classes to the first level of import for easier access and better modularity.

  • Other bug fixes.

Version 0.1.0#

  • Packaged the code for the generation of exiobase 2016 tables.