Home:ALL Converter>from java objects to parquet file

from java objects to parquet file

Ask Time:2016-12-25T23:33:32         Author:TalO

Json Formatter

I have a json file and want to convert it to parquet file. I know how to convert from json file to java objects but still need the conversion from java objects to parquet file. I don't want to convert using AVRO/ Protocol Buffers/Thrift/etc.. and also, don't want to use the Apache drill- just a java code.

Im using Java project with maven, my pom.xml look like this:

<dependencies>
        <dependency>
            <groupId>org.apache.parquet</groupId>
            <artifactId>parquet-common</artifactId>
            <version>1.8.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.parquet</groupId>
            <artifactId>parquet-encoding</artifactId>
            <version>1.8.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.parquet</groupId>
            <artifactId>parquet-column</artifactId>
            <version>1.8.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.parquet</groupId>
            <artifactId>parquet-hadoop</artifactId>
            <version>1.8.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-common</artifactId>
            <version>2.2.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-core</artifactId>
            <version>0.20.2</version>
        </dependency>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-client</artifactId>
            <version>2.2.0</version>
        </dependency>
    </dependencies>

Means I don't have a problem using hadoop.

So I found this link (the last comment):

https://groups.google.com/forum/#!topic/parquet-dev/lfWanFOc040

The above link has what I want, but I have a problem with 2 imports there and can't find the correct dependencies for them.

Those are the 2 imports I can't find:

import org.apache.hadoop.hive.ql.io.parquet.writable.BinaryWritable;
import org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriteSupport;

will appreciate your help for finding the dependencies.

THANX:)

Author:TalO,eproduced under the CC 4.0 BY-SA copyright license with a link to the original source and this disclaimer.
Link to original article:https://stackoverflow.com/questions/41322416/from-java-objects-to-parquet-file
yy