Setup Flink Development Environment

GitHub repo: https://github.com/JingGe/101

Flink provides very comprehensive document you might need to develop Flink or to use Flink to develop your own application. This document will focus on pointing out the most important steps and hint to get the environment ready. The goal is clear: save your time as more as possible.

This document is valid for Flink 1.13. It took me more than two days to get everything done. Hope the estimated time could be optimized to less than 3 hours, if you follow this guide, which means a 500% improvement wrt the time cost.

Goal of this document is to save 90% of the time you spend to get the environment ready for your development.

Get the code and build Flink

To save your time, Java 11, Intellij Idea, and maven 3.8.6 are recommended

Make sure you have Java 11 and Maven 3.8.6 installed. Fork your own Flink repo and checkout the source code from Github and run:

$ mvn clean install -DskipTests

If you are behind firewall, e.g. working in China, and have issue to access some dependencies, you might consider using mirror in your maven settings.xml, for example in China:

    <mirror>
      <id>nexus-aliyun</id>
      <name>Nexus aliyun</name>
      <url>http://maven.aliyun.com/nexus/content/groups/public</url>
    </mirror>

You can read the official doc from Flink for further information in details:

Setting up a Flink development environment

Building Flink from Source

Since the maven build will take long time to download dependencies for the first time, you might consider reading the contribution doc in parallel.

Setup Intellij Idea

Open Flink as a project is easy but the IDE setup of Intellij Idea will take about one hour for the first time**:** Importing Flink into an IDE.

The setup of code formatting and Save Actions are important. It will avoid messing the code up. Worth doing it!

I had got stuck at: 8. Build the Project in Intellij Idea (“Build” → “Build Project”) for java 11 issue because only java 8 was installed on my computer. It took me hours to fix it. After searching and reading a lot of docs and blogs, finally found the right info at FAQ section. Problem solved after setting the project SDK and module SDK to java 8, unchecking the java11 maven profile and reloading all maven projects.

Currently, it is recommended to use Java 8. The migration to Java 11 is under construction.

Now you should have a clean Flink development env ready on your local development. \

---