PPF-based HSUT disaggregation procedures#

The purpose of this document is to provide guidelines for integrating Parameterized Production Function (PPF) data into the Hybrid Supply–Use Table (HSUT) workflow.

For background on the HSUT workflow, please refer to the methodology.

PPF data must be delivered in a structured format consisting of several datasets, as illustrated below:

These datasets represent the ideal PPF output. However, contributors are not required to provide all of them.

Classification#

Before integrating PPF data, the following classification and correspondence tables must be updated or expanded:

  • BONSUT Activity classification

  • BONSUT Product classification

  • BONSAI-BONSUT product correspondence

  • BONSAI-BONSUT activity correspondence

  • BONSUT Monetary SUT to BONSUT physical SUT activity correspondence

  • BONSUT Monetary SUT to BONSUT physical SUT product correspondence

  • BONSUT product-to-market correspondence

If classifications are not yet updated, you should first perform the classification expansion.

Integration and disaggregation#

General principle#

The HSUT workflow consists of multiple registered tasks (see task registration. In most cases, PPF data do not require new tasks. The HSUT workflow loads and integrates all available data sources (LOADED data). Since PPF data are considered more reliable than estimated data, they must take priority when merging. In practice, this means that PPF data may overwrite other data in the workflow.

In other less trivial circumstances, a new task might be necessary to integrate the output of PPFs. This will be discussed in the the following sessions. It will be addressed the integration of one account pert time.

Create a new branch and chage configuration data#

Before working on the BONSAI workflow it is important to create a new branch and to modify the configuration file.

It is important work for the integration of a PPF in a new branch. Only when when the code has been tested and is reliable, it is advised to make a merge request (MR) to add the new code to the dev branch.

Secondly, it is necessary to change the date in the configuration file. By doing so, it is avoided that new exported files might overwrite existing files causing issues to the main workflow.

PPF-based supply table#

  • The supply table includes principal products, by-products and wastes supplied by PPF-modeled processes.

  • Other outputs (e.g. emissions) are treated separately in later sections.

  • PPF-SUT data are always expressed in absolute values.

  • Integration point: task_load_ppf_supply and task_disaggregate_supply_w_ppf.

Steps in the tasks:

  • Load the PPF dataframe.

  • Harmonize regions – aggregate or disaggregate to BONSUT regions using the classifications repository.

  • Harmonize product codes – convert BONSAI codes into BONSUT codes (e.g. C_COKE) for products already present in BONSUT.

  • Handle multiple producers – if more than one activity produces the same product, generate a national market.

  • Standardize format – rename headers to match the base HSUT schema and add time and flag columns.

  • Identify disaggregated processes – match PPF data against the base hybrid supply table to determine which processes are split by PPFs.

  • Generate distribution keys – build coefficients from PPF data for allocating supply among disaggregated processes.

  • Handle unmatched supplies – if a base supply has no direct match in the PPF table, distribute it using the average coefficients derived from related child PPFs.

PPF-based Use table (PPF-USE)#

The PPF-USE table represents:

  • Inputs of products to PPF-modeled activities (recipes).

  • Uses of products produced by PPF-modeled activities (usage of productions).

Example: If a PPF models mobile phone production:

  • The inputs (recipes) include plastics, glass, batteries, and components used in manufacturing.

  • The outputs (usage) describe who consumes the phones (e.g. industries, households, exports).

Recipes and usage are documented separately below. Both must be provided for full integration.

  • If usage data are missing, a workaround procedure (national mix construction) must be applied.

  • All PPF-USE data must be in absolute values.

Recipes#

Recipes include only input of products to activities modelled in the PPF. Use of resources or waste are addressed in sections below. Inputs of packaging has to be inserted in this section only if the packaging good is used by the activity to wrap its productions.

A general rule of PPFs is that the mix of fuels used to produce heat is assigned in the HSUT workflow. Therefore, recipes should only include the total use of heat.

The following procedure is valid only for the default case. If the contributer wants to define a specific heat source, she first needs to convert the energy source into combustion activities and then apply the following procedure.

The conversion of energy source into combustion activities is explained here.

Recipes can be directly integrated in the task:

  • task_uses_before_monetary_distr_keys

  1. load the dataframe.

  2. (temporary) Aggregate/dissagregate to the regions used in bonsai. For that use the classifications repository.

  3. (temporary) convert the fi codes into the “exiobase” ones (e.g. C_COKE) for the products that already existed in exiobase. This would introduce a minimal amount of new codes into the workflow. Aggregate the duplicate rows generated during product aggregation.

  4. rename the headers to match the content of the existing code. Add time and flag columns

  5. go to section: # concatenate data. Update the priority list as indicated in step 7) in PPF-based supply table

An example can be found the function below, that should run sucessfully after the changes

from bonsai.io import Config as HSUTConfig
from bonsai.io.partial_SUTs.create_an_initial_HUSE_def import merge_uses_before_using_monetary_distr_keys

default_config = HSUTConfig()
merge_uses_before_using_monetary_distr_keys(default_config)

Usage of productions#

Differently from the recipes, the task of integrating usage of production might not be trivial because PPF might require a disaggregation of the existing product classification. The following decisional diagram shows the logic to apply when implementing the usage of productions.

The following sub-sections explain how to behave in different situations. Notice that the usage of production data (or simply usage data) may require that one or more of the following steps are adopted at the same time. It all depends on data availability.

Case 1: data on usage are provided#

If the provider of PPF also delivers of who is using the supply of PPF-activities, then the remaining task is to add these values to the USE table. This can be done following the instructions in the above section Recipes.

However, the activity classification of the users has a different disaggregation of BONSAI, a re-distribution has to be performed. #TODO: reference to text somewhere else

Case 2: data on usage are not provided but are already in the database#

If the supply of PPF-activities follows the same product classification adopted in BONSAI, there is not a strong request to make any action. The BONSAI workflow has already the information on who is using the product supplied.

Case 3: data on usage are not provided and there is no trade data#

Because there is no information on who is using the products disaggregated with PPF, a national mix (or export market) has to be build. In that case all the user of the products will use the same proportional mix, regardles if they are located inside or outside the reference country.

The creation of national (production) mixes requires more tasks to be accomplished. They are liste in task registration

An example of national mix construction is here

Construction of the mix:

  • task_domestic_production_fish

Add the national mix to the Supply table

  • task_add_national_mixes_to_supply_matrix

—– TO BE continued#

national mixes#

If in the supply data we have more than one way of producing a product we need to generate a national mix. First we write a function to generate the national mix. examples can be found in bonsai.io.markets.domestic_prod_mix.domestic_production_coke. There are a number of steps required (note that the first ones are identical to those done for the supply table)

  1. load the supply data from the PPF resource

  2. aggregate to bonsai locations

  3. rename products to “old” BONSAI product codes

  4. rename activities to “old” BONSAI activity codes

  5. select the rows refering to the reference product

  6. replace the name of the activity by the name of the national mix (e.g. A_Nat_COKE)

  7. rename the headers to match the rest of the tables.

Then we need to add these national mixes to the supply matrix. To do this we need to modify the function add_national_mixes_to_supply_matrix and

  1. load the dataframe generated in the previous step

  2. add it to the priority list

After the introduction of the new data the function should run sucessfully

from bonsai.io import Config as HSUTConfig
from bonsai.io.partial_SUTs.create_an_initial_HUSE_def import add_national_mixes_to_supply_matrix
default_config = HSUTConfig()

add_national_mixes_to_supply_matrix(default_config)

Calcultion of the residual activity#

Emissions#

In this context, Emissions refers to the total process-related emissions reported in absolute values.

Note

Emissions resulting from the combustion of fuels (e.g. heat and power generation) are not included here, as these are already handled separately within the HSUT workflow.

In order to insert emissions, go to task task_load_ppf_emission.

  1. Load the dataframe of process emissions coming from the PPF.

  2. Map locations to bonsai locations

  3. Map products to BONSAI product codes

  4. Map activities to BONSAI activity codes

  5. Rename headers so they are consistent with the format expected by the existing HSUT tables

If implemented correctly, the function should still run successfully and produce a PPF emissions dataset in BONSAI classifications.

Resource data#

Waste supply data#

Waste use data#

Use of productions supplied by processes modelled in PPFs#