Question:
Consider the following:
- A table in Redshift called ‘people’ that has fields id, name and age
- A kinesis firehose stream called ‘people’ that is configured to write to the ‘people’ table and the value for ‘Redshift table columns’ is ‘id,name,age’
It’s not clear how to format the ‘Data’ blob. Here’s an example of what the code looks like with the data separated by tabs:
|
1 2 3 4 5 6 7 8 9 10 11 |
let AWS = require('aws-sdk'); let firehose = new AWS.Firehose(); let params = { DeliveryStreamName: 'people', // id,name,age Records: [{Data: '4ccf6d3a-acdf-11e5-ad54-28cfe91fa8f1\tBob\tSmith'}] }; firehose.putRecordBatch(params, (err, result) => { console.log(err || result); }); |
Here are some of the docs I have checked:
Answer:
The answer is here:
http://docs.aws.amazon.com/redshift/latest/dg/copy-parameters-data-format.html
Fields need to be pipe ‘|’ separated by default. Rows should be separated by new lines.
Updated corrected code:
|
1 2 3 4 5 6 7 8 9 10 11 |
let AWS = require('aws-sdk'); let firehose = new AWS.Firehose(); let params = { DeliveryStreamName: 'people', // id,name,age Records: [{Data: '4ccf6d3a-acdf-11e5-ad54-28cfe91fa8f1|Bob|Smith\n'}] }; firehose.putRecordBatch(params, (err, result) => { console.log(err || result); }); |