Import tool

This is a tool which can be used to easy importing data into a Uni-d application. The program scans a directory for import files to appear (very configurable) and imports them using ImportServlet. There is special handling to allow errors to be caught and corrected. Alternatively, this program can also execute scripts which can be used to build the import file at runtime (thus allowing a direct link with JDBC data sources etc). This option is typically used to synchronize with another application (one-way synchronization, use the Export tool for the other direction).

Usage

The import tool can either scan directories for import files to appear, or can run scripts which produce the import file (probably by reading data from a proprietary system).

The file format for importing is ImportFileFormat.

General configuration

The tool (class be.unid.tool.ymport.ImportClient) accepts one optional parameter, the name of an .ini file which is used for the configuration. If no parameter is given, the program tries to open "import.ini".

The .ini file has the following format

[general]
    scanPeriod=30
    outputExtension=err
    window=display
    search=local
    importURL=http://192.168.0.252:8080/admin/import/import
    [scan]
    dir1=c:import
    input1=*.txt
    charset1=ISO-8859-1
    dir2=g:import
    input2=*.import
    script2=import.isc
    [script]
    debug=false
    script1=weight.isc
    script2=article.isc
	[database]
	name1=vlees
	driver1=sun.jdbc.odbc.JdbcOdbcDriver
	location1=jdbc:odbc:CodeBase Datasource;database=C:dbf
	user1=
	password1=

This has the following meaning

• General section
• scanPeriod : time in seconds between attempts to pick up files to import or attempt to run the import scripts. Defaults to 60s.
• outputExtension : extension for output (error) files. The extension of the input file is replaced, or the extension is added if the input file had no extension.
• window : indicates whether a window needs to be displayed or not (values "display" or "hidden").
• search : indicates whether the program should only scan in the scan directories, or also in subsidrectories. Possible values are: "deep" and "local".
• Scan section
• This can contain the directories which have to be scanned for input files (scanX with X a number) and a regular expression (inputX) indicating the files which need to be imported (default is *), for the directory with the same number
• It may also contain the charset to be used for the files from the matching directory. When charset is not set, the system default is used.
• For each level, an import script file can be specified. The "build(string filename)" function is called in this script to allow conversion from other types of files.
• Script section
• debug : can be "true" or "false". Normally the data which is built is immediately sent to the import servlet. However, when debug is "true", then the data is not imported but simply written to an output file in the current directory.
• scriptX (with X a number) : name of a script file which builds the data to be imported.
• database section
• As with script and scan, the names need to end with a number to indicate what belongs together. Counting starts at one and should be continuous. The database section is only relevant for import scripts.
• name : name for the database, as used in "Dbset(var,name)"
• driver : JDBC driver class
• location : database url for connection with the database
• user, password : username and password for the connection
• config section
• In the config section extra kay/value pairs can be put which can be retrieved in the scripts using "config(key)".

Program functioning

The import itself starts by computing the name of the error file (the extension (everything after the last dot, if any) is replaced by "outputExtension"). If the error file exists, the import file is skipped. It then tries to open the import file in read/write mode. If that fails, the file is skipped (it will be picked up again in the next scan). When the file is opened, the contents is sent to the import servlet. If the import was successful, then the input file is deleted. If the import failed, the return message from the servlet is written in the error file.

Script handling

The program accepts FScript (see http://fscript.sourceforge.net/index.html) scripts to build the data to be imported. The script should have the following structure :

func build()

this functions is called to builds the import file
You have the following commands for this
table(finderName, commaSeperatedFields)
field(data) : add a field value
line([string]) : next line or add line with given text
quote(string) : quote a string using single quotes, doubles the single quotes in the string
import() : import the current data, returns whther the import was succesful
      (allows forcing import halfway through the script)
if references need storing,
use the "list" variable (of type "List")
endfunc
func onSuccess()

this function is called when the import was succesful
can be used to mark te data as imported
(again, referencing the contents of "list")
endfunc
func onError(string error)

this function is called if the import failed
the "list" data is still available
endfunc

The FScript extensions from the util module are available.