SAP BW Master Data Text Loading Steps

Loading text records is similar to (yet simpler than) loading attribute records, because there are no additional SID tables to update:

Master Data Text Update Algorithm

*** STEP 1 - Process data package record by record

Loop at data package. 

*** STEP 2 - Surrogate key handling

  Retrieve/Generate surrogate key.

  Update SID table.

*** STEP 3 - Non time dependent master data texts.

  Retrieve existing non time dependent record.

  Insert/Update non time dependent texts.

*** STEP 4 - Time dependent master data texts.

  Retrieve existing time dependent records.

  Check for validity period overlaps.

  Adjust overlap records accordingly.

  Insert/Update time dependent texts.

End.

Posted by Kevin McDonald at 01:41 PM | Permalink | Comments (0) | TrackBacks (0)

SAP BW Master Data Attribute Update Steps 

The following steps show a highly simplified master data attributes update process:

Master Data Attribute Update

*** STEP 1 - Process data package record by record

Loop at data package.

*** STEP 2 - Surrogate key handling

  Retrieve/Generate surrogate key.

  Update SID table.

*** STEP 3 - Non time dependent master data.

  Retrieve existing non time dependent record.

  Insert/Update non time dependent attributes.

  Retrieve surrogate keys for attributes.

  If not all SIDs available.

    If master data check requested.

      Log error, start error handling.

    Else.

      Generate new SID.

    End.

  Insert/Update attribute SID table

*** STEP 4 - Time dependent master data.

  Retrieve existing time dependent records.

  Check for validity period overlaps.

  Adjust overlap records accordingly.

  Insert/Update time dependent attributes.

  Retrieve surrogate keys for attributes.

  If not all SIDs available.

    If master data check requested.

      Log error, start error handling.

    Else.

      Generate new SID.

    End.

  Insert/Update attribute SID table

End.

Posted by Kevin McDonald at 01:24 PM | Permalink | Comments (0) | TrackBacks (0)

SAP Data Extraction Steps 

The following is a sample step-by-step data extraction control flow for SAP systems:

   1.  An InfoPackage is scheduled for execution at a specific point of time or for a certain system- or user-defined event.

   2.  Once the defined point of time is reached, the SAP BW system starts a batch job that sends a request IDoc to the SAP source system.

   3.  The request IDoc arrives in the source system and is processed by the IDoc dispatcher, which calls the BI Service API to process the request.

   4.  The BI Service API checks the request for technical consistency. Possible error conditions include specification of DataSources unavailable in the source system and changes in the DataSource setup or the extraction process that have not yet been replicated to the SAP BW system.

   5.  The BI Service API calls the extractor in initialization mode to allow for extractor-specific initializations before actually starting the extraction process. The generic extractor, for example, opens an SQL cursor based on the specified DataSource and selection criteria.

   6.  The BI Service API calls the extractor in extraction mode. One data package per call is returned to the BI Service API, and customer exits are called for possible enhancements. The extractor takes care of splitting the complete result set into data packages according to the IDoc control parameters. The BI Service API continues to call the extractor until no more data can be fetched.

   7.  The BI Service API finally sends a final status IDoc notifying the target system that request processing has finished (successfully or with errors specified in the status IDoc).

Posted by Kevin McDonald at 01:18 PM | Permalink | Comments (0) | TrackBacks (0)

Defining a DataSource for a DB Connect source system follows similar steps as defining a DataSource for the BI Service API:

   1.  A source system must be created with a logical source system name but with additional details such as the database management system (DBMS), the DB user ID and password, and connection string info (DB-dependent parameters).

   2.  A DataSource is created against the DB Connect source system. From the extraction parameters for the database table adapter, the DB Connect interface retrieves a list of tables and views available from the remote database. After an entry is selected from this list field, information for the selected table or view is retrieved from the database catalog. The selected fields are then used as a basis for generating the DataSource definition, which is stored in the Meta Data Repository. The system will propose a list of type conversions from the DataSource definition. You can preview a sampling of what the data will look like in SAP.

After that, you can assign the DataSource to act and behave as any other DataSource in SAP BW.

Posted by Kevin McDonald at 02:22 PM | Permalink | Comments (0) | TrackBacks (0)

Step-by-step control flow for a successful data extraction with SAP BW:

   1.  An InfoPackage is scheduled for execution at a specific point of time or for a certain system- or user-defined event.

   2.  Once the defined point of time is reached, the SAP BW system starts a batch job that sends a request IDoc to the SAP source system.

   3.  The request IDoc arrives in the source system and is processed by the IDoc dispatcher, which calls the BI Service API to process the request.

   4.  The BI Service API checks the request for technical consistency. Possible error conditions include specification of DataSources unavailable in the source system and changes in the DataSource setup or the extraction process that have not yet been replicated to the SAP BW system.

   5.  The BI Service API calls the extractor in initialization mode to allow for extractor-specific initializations before actually starting the extraction process. The generic extractor, for example, opens an SQL cursor based on the specified DataSource and selection criteria.

   6.  The BI Service API calls the extractor in extraction mode. One data package per call is returned to the BI Service API, and customer exits are called for possible enhancements. The extractor takes care of splitting the complete result set into data packages according to the IDoc control parameters. The BI Service API continues to call the extractor until no more data can be fetched.

   7.  The BI Service API finally sends a final status IDoc notifying the target system that request processing has finished (successfully or with errors specified in the status IDoc).

Note

Control parameters specify the frequency of intermediate status IDocs, the maximum size (either in kilobytes or number of lines) of each individual data package, the maximum number of parallel processes for data transfer, and the name of the application server to run the extraction process on.

Posted by Kevin McDonald at 02:14 PM | Permalink | Comments (1) | TrackBacks (0)

Gone but not forgotten:  Transfer Rules

Transfer rules are available only for classic DataSources in SAP BW; for new DataSources, transformations are used instead.

SAP BW offers several ways to define a transformation:

    *  Simple field assignments, where a field of the transfer structure is assigned to a field of the InfoSource

    *  Constant value assignment, where a constant value is assigned to a field of the InfoSource

    *  Formulas, where predefined transformation functions can be used to fill a field of the InfoSource

    *  Routines, which allow you to implement custom ABAP code for complex transformations

Posted by Kevin McDonald at 10:30 AM | Permalink | Comments (0) | TrackBacks (0)

Data-quality management focuses on checking data for the following:

    *  Invalid technical data formats — Invalid or unrecognized date formats or fields with numeric values that actually contain letters or special characters.

    *  Invalid values — Data fields with values out of the range of admissible values according to the definition of the InfoObject.

    *  Missing information — Data fields that do not have a value or have an initial value.

    *  Referential integrity — Data fields with a reference to another data record that cannot be resolved (for example, a customer number that cannot be found in the customer master).

    *  Plausibility — For example, a birth date more than 120 years ago in your current customer master.

Posted by Kevin McDonald at 10:19 AM | Permalink | Comments (0) | TrackBacks (0)

Data integration has multiple aspects

    *  Technical integration — Ensures that the same data formats are used by converting data types, performing basic calculations, and basic data-quality and plausibility checks (such as adding missing information using defaults or derived values).

    *  Semantic integration — Ensures that a common coding of values is used across all objects in the EDW (for example, that the gender is always represented as “M” for male, “F” for female, or “initial” for unknown). While the gender example is a simple one, implementing semantic integration can become very expensive in heterogeneous environments.

    *  Process integration — In many cases, business processes and their respective operational systems have to be enhanced or modified to fully support the requirements for integrated data in the EDW. For example, if a customer group is not maintained on a certain sales system, while it is crucial for analysis purposes, that system and the corresponding business processes must be enhanced to enforce maintenance of customer groups.

    *  Analysis integration — Ideally, the same objects or objects derived from one another are used in all reporting and analysis processes across all systems of the topology to ensure maximum integration.

Posted by Kevin McDonald at 10:16 AM | Permalink | Comments (0) | TrackBacks (0)