Why not move on to responsible and sustainable development?

Posted by dpaven on 3 November 2020 ABAP

It's well known that good development must take into account maintainability (clear, standardized, commented code), but it's less common for resource management to be taken into account, i.e. :

1. The total processing time
2. The number of request to the database
3. Memory consumption on the application server

1/ Processing time

It will only attract attention if it is detrimental to the user or if it does not fit into the production plan. However, just because the execution time seems acceptable does not mean that it is.
For example, if a department spends half a day each week manually making a very important synthetic report in Excel and you deliver a development to them that does it in 1 hour, they will be delighted and won't ask if the 1 hour is justified. But maybe the development is badly done and could have given the same result in 1 min?

In addition, a new development that processes new data may have a good response time because the amount of data to be processed is still small. But in a year or two, it may become excessively slow.

Why is it important to take this into account?

Unnecessarily long processing time is a waste of CPU time at the application server level, which is often accompanied by poor database access management.

How to proceed?

Launch the program with a large amount of data to process and use the SAT transaction to perform runtime analysis. Pay attention to the part that take the most time but also to individual database accesses that are significantly long (you may have SELECT that does not use neither the primary key nor an index when it could).

2/ Quantity of requests to the database

Each request takes a minimal incompressible time and occupies an communication channel with the database.
If you retrieve a million rows with a single SELECT, it will be much faster and more efficiently processed by the database than an application loop with a million SELECTs. This seems obvious, but we often see this 2nd case in developments, whether in straightforward (a SELECT / ENSELECT or a SELECT in a LOOP loop) or more hidden ways (a loop that contains calls - PERFORM, CALL FUNCTION - that make unitary accesses to the database).
A classic solution is to load in internal tables (thus in the application memory) all the necessary data with a minimum of database request and then work only with the internal tables.
The limitation of this solution is the available memory. You don't want to replace a database I/O bottlenecks with a memory overflow.

How to proceed?

Use the Code Inspector to identify all problematic database accesses.
Use the Run Time Analysis (SAT) to spot SELECTs that are too long.

3/ Memory consumption

It is usual to put a very high or no limit on the usable memory for background processing and to put a fairly low limit for foreground processing. Indeed, a background processing releases the memory at the moment it has finished its processing, while in the foreground the memory will only be released when the user exits his transaction (or logs off if the associated work process has used Heap Memory and has switched to PRIVATE mode). Without this protection, a handful of users launching very greedy states and leaving them open could monopolize all the server's memory.

Consequences :

• Programs used in the background can waste a huge amount of memory without being noticeable. Until the day when several of them running at the same time will go to a memory overflow and dump.
• Programs used in the foreground handling a lot of data may be interrupted due to lack of memory. In the best case, an incident will be opened, in the worst case users who can do it will just run the states in the background.

How to limit the amount of memory used?

a) Only retrieve the necessary fields

It's easy to create a structure like the source table and then make a SELECT *. But it will be faster and cheaper in memory to create the structure with only the necessary fields and to bring only those in the SELECT. This also avoids many bugs due to the access to a wrong field with a name close to the expected one.

b) Select only the necessary entries

A SELECT is sometimes followed by a DELETE loop to clean up the selection on an additional criterion.
Make sure that this criterion cannot be added directly in the SELECT.

c) Joints

One is tempted to copy the standard tables into internal tables individually. But if an internal table is only used once in a FOR ALL ENTRIESS, then maybe there was a way to make a single selection with a join.
The power of joins can make it unnecessary to retrieve intermediate information and then saves memory.

d) Freeing up memory

Whenever you no longer need an internal table, remember to free the memory with the FREE instruction. The CLEAR statement clears the data from the table, but does not free the memory.

e) Avoid table copies

Some situations that are frequently encountered in developments :

• You need to make a LOOP loop on a sub-part of an internal table?

Instead of copying it and making a DELETE in the copy, maybe it is possible to make a LOOP with a selection criterion or to make a LOOP with a test to skip unnecessary lines (especially if the part of the table that is deleted is small compared to the size of the table).

• You need to do a FOR ALL ENTRIES on a sub-part of an internal table?

Consider copying only the columns needed for SELECT FOR ALL ENTRIES (often only 1) into a temporary internal table.

• For performance, you are using a sorted table with a unique key and you need to pass it as a parameter to a function module that only accepts STANDARD tables.

You can declare the table as STANDARD and use a sorted unique secondary key.

TYPES: BEGIN OF ty_mara,
       matnr TYPE vbrp-matnr,
       ...
     END OF ty_mara.
DATA lt_mara TYPE STANDARD TABLE OF ty_mara WITH UNIQUE SORTED KEY by matnr COMPONENTS matnr.

f) Using a packet size

You have optimized everything, but you still have a peak in memory usage that exceeds the memory available during the process, while the size of the result remains within limits.
You can set a packet size "n" and use it to limit either the selection of primary data or the data retrieved from this primary data via a SELECT ... INTO ... PACKAGE SIZE n ... / ENDSELECT.
It will be necessary to do some tests to find the right packet size that will reduce memory consumption enough without increasing the execution time too much.

g) Do not load in memory what is already there

When you get into the habit of working with internal tables, sometimes you overdo it.
We must not forget that some tables are buffered and that a SELECT on these tables is as fast as a READ on an internal table. This is especially the case in the standard for parameter tables and some small applicative tables that are very often accessed.
But maybe you also have specific applicative tables that are used in many reports and that have been buffered. You should make sure of this.