I'm using azure SDK, avro-parquet and hadoop libraries to read a parquet file from Blob Container. Currently, I'm downloading file to the temp file, and then create a ParquetReader.
try (InputStream input = blob.openInputStream()) {
Path tmp = Files.createTempFile("tempFile", ".parquet");
Files.copy(input, tmp, StandardCopyOption.REPLACE_EXISTING);
IOUtils.closeQuietly(input);
InputFile file = HadoopInputFile.fromPath(new org.apache.hadoop.fs.Path(tmp.toFile().getPath()),
new Configuration());
ParquetReader<GenericRecord> reader = AvroParquetReader.<GenericRecord> builder(file).build();
GenericRecord record;
while ((record = reader.read()) != null) {
recordList.add(record);
}
} catch (IOException | StorageException e) {
log.error(e.getMessage(), e);
}
I want to read this file using inputStream from azure blob item, without downloading it to my machine. There's such way for S3 ( Read parquet data from AWS s3 bucket), but does this possibility exist for Azure?