5 Populating the IM Column Store Manually
The database does not automatically populate n-Memory objects whose PRIORITY setting is NONE. To populate these objects, you must execute SQL or PL/SQL.
This chapter contains the following topics:
- About Manual Population of In-Memory Objects
Populate objects manually using either a full table scan,DBMS_INMEMORYprogram units, orDBMS_INMEMORY_ADMIN.POPULATE_WAIT. - Forcing Initial Population of an In-Memory Object
You can force population of an object using a full table scan, thePOPULATEprocedure, thePOPULATE_WAITfunction, or theREPOPULATEprocedure. - Populating In-Memory Tables Manually: Examples
The following examples illustrate how to populate In-Memory tables manually.
Parent topic: Configuring and Populating the IM Column Store
5.1 About Manual Population of In-Memory Objects
Populate objects manually using either a full table scan, DBMS_INMEMORY program units, or DBMS_INMEMORY_ADMIN.POPULATE_WAIT.
If you enabled an object with PRIORITY set to NONE, and if you want to populate it immediately, then you have the options described in the following sections:
- Population Using SELECT
You can initiate population by issuing aSELECTstatement that forces a full table scan. - Population Using DBMS_INMEMORY.POPULATE
TheDBMS_INMEMORY.POPULATEprocedure achieves the same goal as a full scan. - Population Using DBMS_INMEMORY_ADMIN.POPULATE_WAIT
TheDBMS_INMEMORY_ADMIN.POPULATE_WAITfunction initiates population of allINMEMORYobjects that have a priority greater than or equal to the specified priority, and returns a status value for the population. A user-specified interval specifies the maximum time that the function waits before returning the value to the caller. - Population Using DBMS_INMEMORY.REPOPULATE
DBMS_INMEMORY.REPOPULATEforces repopulation of a table, partition, or subpartition that is currently populated in the IM column store.
Parent topic: Populating the IM Column Store Manually
5.1.1 Population Using SELECT
You can initiate population by issuing a SELECT statement that forces a full table scan.
In this case, the database reads each row in the object and converts it to columnar format. Note that the following statement does not guarantee a full table scan:
SELECT COUNT(*) FROM objectThe reason is that the optimizer may choose to scan an index. Therefore, Oracle recommends forcing a full table scan by using the FULL hint for SELECT COUNT(*) queries, as in the following example:
SELECT /*+ FULL(customers) NO_PARALLEL(customers) */ COUNT(*) FROM customers;See Also:
-
"Populating an In-Memory Table Using a Full Table Scan: Example"
-
Oracle Database SQL Language Reference to learn more about the
FULLhint
Parent topic: About Manual Population of In-Memory Objects
5.1.2 Population Using DBMS_INMEMORY.POPULATE
The DBMS_INMEMORY.POPULATE procedure achieves the same goal as a full scan.
The database reads every row in the specified object, converts it from row format to columnar format, and then populates it in the IM column store. The following PL/SQL block initiates population of the customer table:
BEGIN
DBMS_INMEMORY.POPULATE( schema_name => 'SH', table_name => 'CUSTOMERS');
END;
/See Also:
-
Oracle Database PL/SQL Packages and Types Reference to learn about the
DBMS_INMEMORYpackage
Parent topic: About Manual Population of In-Memory Objects
5.1.3 Population Using DBMS_INMEMORY_ADMIN.POPULATE_WAIT
The DBMS_INMEMORY_ADMIN.POPULATE_WAIT function initiates population of all INMEMORY objects that have a priority greater than or equal to the specified priority, and returns a status value for the population. A user-specified interval specifies the maximum time that the function waits before returning the value to the caller.
Sample use cases for ensuring that objects are populated include:
-
When the database is closed, open the database with
STARTUP RESTRICTso that only administrators can access the database, and then executePOPULATE_WAITwith the desired timeout setting. When the function returns0, disable the restricted session so that non-administrative users can query the database. -
Block database connections by using services or an application tier technique. When no analytic indexes exists, and when the application depends on the IM column store to provide reasonable performance, these techniques prevent runaway queries.
The POPULATE_WAIT function does not accept a table name as input. Rather, the function processes all INMEMORY objects with a PRIORITY setting greater than or equal to the priority specified (the default is LOW). If priority is NONE, then the function processes all INMEMORY objects. Note that POPULATE_WAIT does not apply to external tables, which have no priority setting.
The function accepts a population percentage as input, which defaults to 100, and a timeout interval, which defaults to 99999999 seconds (115.74 days). When you execute the function, the database attempts to populate the objects that meet the specified PRIORITY criteria within the timeout interval, and then returns a value indicating the population status.
The following table describes the possible return values for POPULATE_WAIT. The function returns the values 0, 1, 2, and 3 only if the condition is met before the end of the interval specified by timeout. For example, if timeout is 600, then the function returns 1 only if an out-of-memory error occurs before 600 seconds pass. The function returns -1 when the end of the timeout interval occurs before the database completes the requested operation.
Table 5-1 Return Values for POPULATE_WAIT
| Constant | Value | Description |
|---|---|---|
|
|
|
The function timed out while waiting for population to complete. |
|
|
|
All objects that met the |
|
|
|
The In-Memory pool had insufficient memory to populate the objects that met the |
|
|
|
No |
|
|
|
The In-Memory column store is not enabled. |
See Also:
-
"Setting a Timeout Using the POPULATE_WAIT Function: Example"
-
Oracle Database PL/SQL Packages and Types Reference to learn about the
DBMS_INMEMORY_ADMINpackage
Parent topic: About Manual Population of In-Memory Objects
5.1.4 Population Using DBMS_INMEMORY.REPOPULATE
DBMS_INMEMORY.REPOPULATE forces repopulation of a table, partition, or subpartition that is currently populated in the IM column store.
If you use this procedure on an In-Memory object that is not currently populated, then this procedure is functionally equivalent to DBMS_INMEMORY.POPULATE.
See Also:
-
"Refreshing an In-Memory External Table Using the REPOPULATE Procedure: Example"
-
Oracle Database PL/SQL Packages and Types Reference to learn about the
DBMS_INMEMORYpackage -
Oracle Database SQL Language Reference to learn more about the
FULLhint
Parent topic: About Manual Population of In-Memory Objects
5.2 Forcing Initial Population of an In-Memory Object
You can force population of an object using a full table scan, the POPULATE procedure, the POPULATE_WAIT function, or the REPOPULATE procedure.
Assumptions
This task assumes the following:
-
The IM column store is enabled.
-
You want to enable a table for In-Memory population.
-
You want to force the immediate population of the table into the IM column store.
To force population of an INMEMORY table:
-
In SQL*Plus or SQL Developer, log in to the database as a user with administrative privileges.
-
Apply the
INMEMORYattribute to the table.For example, enable
sh.customersfor IM population as follows:ALTER TABLE sh.customers INMEMORY;In the preceding example, the default priority is
NONE. -
Optionally, check the population status by querying
V$IM_SEGMENTS.For example, use the following statement (sample output included):
COL OWNER FORMAT a10; COL NAME FORMAT a25; COL STATUS FORMAT a10; SELECT OWNER, SEGMENT_NAME NAME, POPULATE_STATUS STATUS FROM V$IM_SEGMENTS WHERE SEGMENT_NAME = 'CUSTOMERS'; no rows selectedThe preceding output shows that the object is not yet populated in the IM column store.
-
Initiate population using one of the following techniques:
-
Query all rows in the table using
SELECTwith theFULLhint.For example, issue the following statement:
SELECT /*+ FULL(customers) NO_PARALLEL(customers) */ COUNT(*) FROM sh.customers; -
Execute the
DBMS_INMEMORY.POPULATEprocedure.For example, execute this procedure for
sh.customersas follows:EXEC DBMS_INMEMORY.POPULATE('SH', 'CUSTOMERS'); -
Execute the
DBMS_INMEMORY.REPOPULATEprocedure.For unpopulated tables, this procedure is functionally equivalent to
POPULATE. For example, execute this procedure forsh.customersas follows:EXEC DBMS_INMEMORY.REPOPULATE('SH', 'CUSTOMERS'); -
Execute the
DBMS_INMEMORY_ADMIN.POPULATE_WAITfunction.The following code example populates all
INMEMORYobjects, regardless ofPRIORITYsetting. The example specifies that the function should wait until all objects are 100% populated, and it should time out with an error if success is not achieved within 1800 seconds (30 minutes).VARIABLE b_pop_status NUMBER BEGIN SELECT DBMS_INMEMORY_ADMIN.POPULATE_WAIT( priority => 'NONE' , percentage => 100 , timeout => 1800 , force => FALSE ) INTO :b_pop_status FROM dual; END; / PRINT b_pop_status
-
-
Optionally, to check the population status, query
V$IM_SEGMENTS.For example, use the following statement (sample output included):
SELECT OWNER, SEGMENT_NAME NAME, POPULATE_STATUS STATUS FROM V$IM_SEGMENTS WHERE SEGMENT_NAME = 'CUSTOMERS'; OWN NAME STATUS --- ---------- -------------------- SH CUSTOMERS COMPLETEDThe table is now populated in the IM column store.
See Also:
-
Oracle Database Reference to learn about
V$IM_SEGMENTS -
Oracle Database PL/SQL Packages and Types Reference to learn about
DBMS_INMEMORYsubprograms
Parent topic: Populating the IM Column Store Manually
5.3 Populating In-Memory Tables Manually: Examples
The following examples illustrate how to populate In-Memory tables manually.
This section contains the following topics:
- Populating an In-Memory Table Using a Full Table Scan: Example
This example using a full table scan to populate thesh.salestable into the IM column store. - Populating a Table Using the POPULATE Procedure: Example
This example usesDBMS_INMEMORY.POPULATEto initiate population of thesh.customerstable into the IM column store. - Setting a Timeout Using the POPULATE_WAIT Function: Example
This example usesDBMS_INMEMORY_ADMIN.POPULATE_WAITto populate all In-Memory tables, regardless of priority setting. - Populating an In-Memory External Table Using DBMS_INMEMORY.POPULATE: Example
This example populates an external table that has theINMEMORYoption. - Refreshing an In-Memory External Table Using the REPOPULATE Procedure: Example
This example repopulates a currently populated In-Memory external table.
Parent topic: Populating the IM Column Store Manually
5.3.1 Populating an In-Memory Table Using a Full Table Scan: Example
This example using a full table scan to populate the sh.sales table into the IM column store.
Assume that you are logged in to the database as an administrator, and that you have issued the following DDL statement to add the INMEMORY clause to the sh.sales table:
ALTER TABLE sh.sales INMEMORY;
The preceding statement uses the defaults for the INMEMORY clause: MEMCOMPRESS FOR QUERY and PRIORITY NONE. Because PRIORITY is NONE, the database will not automatically populate the table into the IM column store. The following query confirms that the sh.sales table is not currently populated:
COL OWNER FORMAT a10;
COL NAME FORMAT a25;
COL STATUS FORMAT a10;
SELECT OWNER, SEGMENT_NAME NAME,
POPULATE_STATUS STATUS
FROM V$IM_SEGMENTS
WHERE SEGMENT_NAME = 'SALES';
no rows selectedThe following query uses the FULL hint to force a full table scan of sales, thereby initiating population (sample output included):
SELECT /*+ FULL(sales) NO_PARALLEL(sales) */ COUNT(*) FROM sh.sales;
COUNT(*)
----------
918843
The following query shows the population status of sales (sample output included):
SET PAGESIZE 50000
COL OWNER FORMAT a3
COL NAME FORMAT a10
COL STATUS FORMAT a20
SELECT OWNER, SEGMENT_NAME NAME,
POPULATE_STATUS STATUS
FROM V$IM_SEGMENTS
WHERE SEGMENT_NAME = 'SALES';
OWN NAME STATUS
--- ---------- --------------------
SH SALES COMPLETED
SH SALES COMPLETED
SH SALES COMPLETED
SH SALES COMPLETED
SH SALES COMPLETED
SH SALES COMPLETED
SH SALES COMPLETED
SH SALES COMPLETED
SH SALES COMPLETED
SH SALES COMPLETED
SH SALES COMPLETED
SH SALES COMPLETED
SH SALES COMPLETED
SH SALES COMPLETED
SH SALES COMPLETED
SH SALES COMPLETED
16 rows selected.
The following query calculates the compression ratio of the table. The query assumes that the tables are not further compressed on disk.
COL OWNER FORMAT a5
COL SEGMENT_NAME FORMAT a5
SET PAGESIZE 50000
SELECT v.OWNER, v.SEGMENT_NAME, v.BYTES ORIG_SIZE,
v.INMEMORY_SIZE IN_MEM_SIZE,
ROUND(v.BYTES / v.INMEMORY_SIZE, 2) COMP_RATIO
FROM V$IM_SEGMENTS v
ORDER BY 4;
OWNER SEGME ORIG_SIZE IN_MEM_SIZE COMP_RATIO
----- ----- ---------- ----------- ----------
SH SALES 851968 1310720 .65
SH SALES 835584 1310720 .64
SH SALES 925696 1310720 .71
SH SALES 958464 1310720 .73
SH SALES 950272 1310720 .73
SH SALES 786432 1310720 .6
SH SALES 876544 1310720 .67
SH SALES 753664 1310720 .58
SH SALES 1081344 1310720 .83
SH SALES 901120 1310720 .69
SH SALES 925696 1310720 .71
SH SALES 933888 1310720 .71
SH SALES 843776 1310720 .64
SH SALES 999424 1310720 .76
SH SALES 581632 1507328 .39
SH SALES 696320 1507328 .46
16 rows selected.Parent topic: Populating In-Memory Tables Manually: Examples
5.3.2 Populating a Table Using the POPULATE Procedure: Example
This example uses DBMS_INMEMORY.POPULATE to initiate population of the sh.customers table into the IM column store.
Assume that you are logged in to the database as an administrator, and that you have issued the following DDL statement to add the INMEMORY clause to the sh.customers table:
ALTER TABLE sh.customers INMEMORY;
The preceding statement uses the defaults for the INMEMORY clause: MEMCOMPRESS FOR QUERY and PRIORITY NONE. Because PRIORITY is NONE, the database will not automatically populate the table into the IM column store. The following query confirms that the sh.customers table is not currently populated:
COL OWNER FORMAT a10;
COL NAME FORMAT a25;
COL STATUS FORMAT a10;
SELECT OWNER, SEGMENT_NAME NAME,
POPULATE_STATUS STATUS
FROM V$IM_SEGMENTS
WHERE SEGMENT_NAME = 'CUSTOMERS';
no rows selectedThe following PL/SQL code uses the POPULATE procedure to initiative population:
EXEC DBMS_INMEMORY.POPULATE('SH', 'CUSTOMERS');The following query shows the population status of customers (sample output included):
SELECT OWNER, SEGMENT_NAME NAME,
POPULATE_STATUS STATUS
FROM V$IM_SEGMENTS
WHERE SEGMENT_NAME = 'CUSTOMERS';
OWN NAME STATUS
--- ---------- --------------------
SH CUSTOMERS COMPLETED
Parent topic: Populating In-Memory Tables Manually: Examples
5.3.3 Setting a Timeout Using the POPULATE_WAIT Function: Example
This example uses DBMS_INMEMORY_ADMIN.POPULATE_WAIT to populate all In-Memory tables, regardless of priority setting.
Example 5-1 Specifying a Timeout Interval for In-Memory Population
In this example, the database contains a number of In-Memory tables with a variety of priority settings. Your goal is to populate every In-Memory table to 100% completion in a restricted database session, and then disable the restricted session so that the application can be guaranteed of querying only the In-Memory representations.
Assume that the database is shut down. In SQL*Plus, you connect to an idle instance as SYSDBA, and then execute the following command (sample output included):
SQL> STARTUP RESTRICT
ORACLE instance started.
Total System Global Area 1157624280 bytes
Fixed Size 8839640 bytes
Variable Size 754974720 bytes
Database Buffers 16777216 bytes
Redo Buffers 7933952 bytes
In-Memory Area 369098752 bytes
Database mounted.
Database opened.
The database is open, but is accessible only to administrative users. You execute the following statements in SQL*Plus (sample output shown in bold):
VARIABLE b_pop_status NUMBER
SELECT DBMS_INMEMORY_ADMIN.POPULATE_WAIT(
priority => 'NONE' ,
percentage => 100 ,
timeout => 300 )
INTO b_pop_status
FROM DUAL;
PRINT b_pop_status
-1After 5 minutes, the function returns the number –1. This code indicates that the function timed out while waiting for population to complete. 5 minutes is not long enough to populate all INMEMORY tables. You re-execute the SELECT statement, specifying a 30-minute timeout:
SELECT DBMS_INMEMORY_ADMIN.POPULATE_WAIT(
priority => 'NONE' ,
percentage => 100 ,
timeout => 1800 )
INTO b_pop_status
FROM DUAL;
PRINT b_pop_status
0After 8 minutes, the function returns the number 0. This code indicates that all tables are completely populated. You now disable the restricted session so that the application can start query In-Memory objects with full confidence that only In-Memory representations will be accessed:
ALTER SYSTEM DISABLE RESTRICTED SESSION;Parent topic: Populating In-Memory Tables Manually: Examples
5.3.4 Populating an In-Memory External Table Using DBMS_INMEMORY.POPULATE: Example
This example populates an external table that has the INMEMORY option.
This example assumes that you created the external table sh.admin_ext_sales with the INMEMORY option using the sh_sales.csv file.
The admin_ext_sales table is not yet populated. Starting in Oracle Database 19c, a full table scan populates an external table just as it populates a standard table. However, in this scenario, you choose to initiate population by using the DBMS_INMEMORY package.
Note:
Sessions that query In-Memory external tables must have the initialization parameter QUERY_REWRITE_INTEGRITY set to stale_tolerated. If an external table is modified, then the results from the IM column store are undefined.
To populate the table, you perform the following steps:
-
Log in as user
sh. -
Set
QUERY_REWRITE_INTEGRITYin the database session:ALTER SESSION SET QUERY_REWRITE_INTEGRITY=stale_tolerated; -
Execute the following PL/SQL program:
EXEC DBMS_INMEMORY.POPULATE('SH', 'ADMIN_EXT_SALES'); -
Check the population status by querying
V$IM_SEGMENTS:COL OWNER FORMAT a3 COL NAME FORMAT a15 COL STATUS FORMAT a9 SELECT OWNER, SEGMENT_NAME NAME, POPULATE_STATUS STATUS FROM V$IM_SEGMENTS; OWN NAME STATUS --- --------------- --------- SH ADMIN_EXT_SALES COMPLETEDThe preceding output shows that the
admin_ext_salestable has been populated
See Also:
-
Oracle Database PL/SQL Packages and Types Reference to learn more about
DBMS_INMEMORY.POPULATE -
Oracle Database Reference to learn about the
QUERY_REWRITE_INTEGRITYinitialization parameter
Parent topic: Populating In-Memory Tables Manually: Examples
5.3.5 Refreshing an In-Memory External Table Using the REPOPULATE Procedure: Example
This example repopulates a currently populated In-Memory external table.
/tmp/data/sh_sales.csv to create the sh.admin_ext_sales table with the INMEMORY option, and that you populated this table into the IM column store.
Assume you add a record to /tmp/data/sh_sales.csv as follows:
echo "148,8787,23-NOV-01,2,999,1,23.43" >> /tmp/data/sh_sales.csv-
Call the
DBMS_INMEMORY.REPOPULATEprocedure -
Specify the table as
NO INMEMORY, specify it asINMEMORY, and then perform a full table scan
The following example uses the REPOPULATE procedure to force the IM column store to refresh admin_ext_sales:
EXEC DBMS_INMEMORY.REPOPULATE('SH', 'ADMIN_EXT_SALES');See Also:
-
Oracle Database PL/SQL Packages and Types Reference to learn more about
DBMS_INMEMORY.REPOPULATE
Parent topic: Populating In-Memory Tables Manually: Examples