OK. I can't seem to figure out how to set up a TIMPORT schema file for my data, nor can I determine from the documentation how my data should be delimited in the data files that TIMPORT will read. But since I can be very flexible in how the data is to appear in the data files, I'm just going to describe my table and my situation and let someone guide me (or better yet, just show me).
I've got a table of 90 varchar columns. Only 20 are used, but the unused columns must remain in the table for purposes not relevant here. (That is, I cannot redefine the table so that it does not include the unused columns. They will just have to remain empty.) All of the columns that are used contain data that is arbitrarily long. The data can be in excess of 100K bytes per column per record. (FYI, each record of the table contains the contents of a single text document, where each column contains a section of that document.) The data has double quotes, single quotes, linefeeds, URLS, etc., which must *all* be preserved. I *cannot* alter the data in any way (e.g., by escaping or doubling all of double-quotes, etc.) There is simply too much data to do that (potentially in excess of 100GB!)
For the sake of this example, assume that the names of the columns in the table are MYFLD1, MYFLD2, MYFLD3, ..., MYFLD89, and MYFLD90. Also assume that the table is named MYTABLE.
One requirement: The data files *must* contain multiple records. There will typically be 300-500 records per file, so the data files will be fairly large.
What should my schema file look like? What should my delimiters look like? (It would be nice if the field and record delimiters were on lines by themselves, since that makes the data files easier to read using a simple text editor. But that is not a requirement.) What should the layout of data files look like? As I said, I can use any type of schema file, and I I can lay the data and it's delimiters out anyway that will work. Really, the only requirement is that the data between the delimiters must look exactly like it does in the section of the document from which it was retrieved. I just need something that *will* work.
In my mind the layout of the data files should be as simple as:
------------------------
MYFLD1DELIMITER:
...arbitrarily
long
and spaced MYFLD1 data with "quoted text"...
MYFLD15DELIMITER:
...arbitrarily long and
spaced
MYFLD15 data with more "quoted text"...
MYFLD35DELIMITER:
..arbitrarily
long
and spaced MYFLD35
data...
-----------------
And so on, repeating for record after record. (Notice how the data can and will span many mulitple lines per column per record.)
I just can't figure it out. Any help would be appreciated.
Barry
I've got a table of 90 varchar columns. Only 20 are used, but the unused columns must remain in the table for purposes not relevant here. (That is, I cannot redefine the table so that it does not include the unused columns. They will just have to remain empty.) All of the columns that are used contain data that is arbitrarily long. The data can be in excess of 100K bytes per column per record. (FYI, each record of the table contains the contents of a single text document, where each column contains a section of that document.) The data has double quotes, single quotes, linefeeds, URLS, etc., which must *all* be preserved. I *cannot* alter the data in any way (e.g., by escaping or doubling all of double-quotes, etc.) There is simply too much data to do that (potentially in excess of 100GB!)
For the sake of this example, assume that the names of the columns in the table are MYFLD1, MYFLD2, MYFLD3, ..., MYFLD89, and MYFLD90. Also assume that the table is named MYTABLE.
One requirement: The data files *must* contain multiple records. There will typically be 300-500 records per file, so the data files will be fairly large.
What should my schema file look like? What should my delimiters look like? (It would be nice if the field and record delimiters were on lines by themselves, since that makes the data files easier to read using a simple text editor. But that is not a requirement.) What should the layout of data files look like? As I said, I can use any type of schema file, and I I can lay the data and it's delimiters out anyway that will work. Really, the only requirement is that the data between the delimiters must look exactly like it does in the section of the document from which it was retrieved. I just need something that *will* work.
In my mind the layout of the data files should be as simple as:
------------------------
MYFLD1DELIMITER:
...arbitrarily
long
and spaced MYFLD1 data with "quoted text"...
MYFLD15DELIMITER:
...arbitrarily long and
spaced
MYFLD15 data with more "quoted text"...
MYFLD35DELIMITER:
..arbitrarily
long
and spaced MYFLD35
data...
-----------------
And so on, repeating for record after record. (Notice how the data can and will span many mulitple lines per column per record.)
I just can't figure it out. Any help would be appreciated.
Barry
