|
<< Click to Display Table of Contents >> How to Import Data |
  
|
|
<< Click to Display Table of Contents >> How to Import Data |
|
Two methods can be employed to import the user's data to the data mart system (including local cloud and distributed cloud).
❖Automatic method
Start all nodes in the system, and the log on Client node shall display the available Naming node. Enter the schedule tasks interface on Client node, add a job, and define the task of this job is Incremental Import Data to Data Mart. This task is specially responsible for extracting the data from one Query (data set ) to the mart system. Option indicator:
Query：choose a data set which data will be extracted. This data set is defined through Create Data Set interface, accessing a database.
Folder：put the extracted data block in which folder.
File：use what file as prefix for the extracted data block. If there are two data blocks, the prefix is aa, there will be two physical files aa0.zb and aa1.zb.
Append：whether append files, or the historical file might be deleted. If there is an aa file, a bb file can be appended. If the appended file name exists, the appending will stop.
Script：this script can be executed before running query. This script can revise the value of folder, file and append, also can revise metadata through setMeta/getMeta; and assign values for parameters.
Once a job is executed successfully, through creating data mart data set, the data can be accessed from the file. Through filtering the value of metadata, We can access each data block by cross-granular.
The user can specify the plan in which intervals the data is extracted, and during the extraction, the data blocks are labeled by adding metadata. If adding the data label, the data access of certain time slot can be controlled.
Please see the relevant documents for the detailed instruction of schedule tasks.
❖Manual method
Z product provides some API interfaces to access the database and read data, then generate the compressed file. Data mart system provides some API management command. Including AddFolderTask (add the compressed physical file in a new Folder belonged to could system), RemoveFolderTask (delete certain Folder), RemoveGSFileTask ( delete certain file).