Some Thoughts on OSD700

As a junior developer, you know how to code is adequate. While as a senior developer, you should know how to maintain and manage a project, as well as how to review other’s code. From now on, I will be learning these skills and gaining experience in the field of open source development.

There are two areas that I will be focusing on, project management and technical improvement.

Project management

This is a big challenge of this course. Since I have no experience in managing and maintaining a big project, I don’t know where to start it.

For instance, I reviewed a Pull Request #1527, it failed in Continuous Integration check – Netlify. After doing a research, I found that Netlify is a web deployment platform. But I was confused that we had already included Vercel in CI check, and why we need to include Netlify as well? Another question is, what message I could leave to the author? As I didn’t find any useful error messages on Netlify app, the logs only pointed out “Failing build”. I could not give the author any suggestion on this issue, if I encounter the same problem, I don’t know how to improve my code to pass the check. Therefore, although I have reviewed this PR, it looks like I have not done anything.

Technical Contribution

This weekend, I went through 3 projects.

I definitely want to get involved in Telescope. I am sure I will spend my time on this project.

I gave my contribution to project Blockly last semester. It is written by pure JavaScript, so you don’t need to know numerous frameworks or libraries to get involved. However, it is a full-fledged project, finding an existing bug or issue to fix is not easy. Therefore, I will keep checking out its issues to find something I can do.

I know this project from David’s case study. I am interested in this project and set it up locally, if I have enough time, I think I will give a try on it.

Let’s start a new journey.

Part B on Release0.4

This blog I want to talk about CSS breakpoint and hamburger menu.

CSS breakpoints are points where the web page layout renders according to the device width. It’s also known as media query,  applying styles with the CSS @media.

The figure illustrates three layouts with three different screen width.

In release0.4, I set screen max-width 768px as mobile header version.

Now, I use CSS @media query to implement hamburger menu in one component Header.js. When you click the three-line menu icon, a side navigation shows up. Mobile header looks like:

Problems need to be solved.

  1. The height of sidebar is not the same as the height of the entire page. (solved)
  2. Click anywhere to close the sidebar. (solved)

You can find this part of solution here. I will keep addressing the problems.

Updated on Dec.12, all issues are fixed. The final PR is here.

Part A on Release0.4

Next.js is another open-source React based front-end development framework. My release0.4 task is writing header component for Telescope via Next.js. Because of the preparation for final exams, release0.4 is postponed to this week.

Like React, to add header for all pages, I design a header component. Next.js simplifies the syntax. We can use HTML elements directly in the rendering.

The hard part is Next.js build-in CSS. In Telescope, we use Gatsby and @material-ui to facilitate the design of user interface. Now, we need to migrate the project front-end to Next.js, I am not sure if we will still use @material-ui. Therefore, I create a file Header.module.css . My Header.js component imports styles from this stylesheet.

The biggest problem of CSS is sometime the issue is not caused by what you write. For example, I spend a lot of time to solve a margin problem as shown. Eventually, I realize that this is a padding setting in it’s parent component Home.module.css.

Now, the desktop header looks like:

I submit this approach as a partial solution of issue#1446. The reason is that I am not sure whether we need to use @material-ui or not. I want to check if the design is in the right way.

For Mobile version header, I am using @media breakpoint to implement the feature. I will talk about it in my next blog.

Deploy a Java application

This blog records the steps to publish a Java application to the Central Repository.

First, what is the Central Repository?

It is a platform to publish the applications, maintained by Sonatype Inc. The Central Repository is the default repository for Apache Maven, Gradle etc. Maven Central Repository Search is the official central search engine of the Central Repository, whereas Central Repository: (apache.org) is the real repository where your component is published.

For example, you can download my application from Maven Central Repository Search directly, while you can find the source files from Central Repository: com/github/jossiey/LinkChecker-CmdLC/1.0 (maven.org) as well.

Next, this blog is focusing on the deployment of an open source individual projects. The official instructions can be found here: Individual Projects – Open Source Software Repository Hosting (OSSRH). It provides a number of short tutorials for beginners, which are helpful.

Now, let’s start to publish a Java application.

Step1: Create a ticket with Sonatype

Before you can upload your project to Nexus Repository Manager (sonatype.org), you need to:

  1. Create a Sonatype JIRA account
  2. Create a New Project ticket

If you choose com.github.your_github_account as your Group Id, you will receive an email asking you to create a public repo to verify github account ownership. Once you done, you are ready to next step.

Step2: Generate a PGP Signature

  1. Install GnuPG

Download GPG from http://www.gnupg.org/download/ or install it with your favorite package manager. Then run gpg --version to verify it works.

2. Generate a Key Pair

Run command gpg --gen-key , follow the prompt entering your name, email, and a comment for the key.

Run command gpg --list-keys to list your public keys

3. Distribute the Public Key

gpg2 --keyserver hkp://pool.sks-keyservers.net --send-keys 435EF2D3

This command distributes your public key to a key server. Note: You don’t need to sign a file now, once you configure your pom.xml file, mvn will do the sign for you automatically.

Step3: Configure your pom.xml and settings.xml files

  1. Configure pom.xml
  • Distribution Management and Authentication
  • Project Profile
  • Nexus Staging
  • Javadoc and Source Attachments
  • GPG Signed Component

This component will sign your project_name.jar, project_name-javadoc.jac, project_name-sources.jar with your public key automatically when you run mvn deploy. You can find the signature files project_name.jar.asc, project_name-javadoc.jac.asc, project_name-sources.jar.asc in the target folder.

2. Configure settings.xml

In Eclipse IDE, open your settings.xml from Windows -> Preference -> Maven -> User Settings.

Alternatively, you can open the file from the directory.

Global Setting C:\Program Files\apache-maven-3.6.3\conf\settings.xml   [global settings]
User Setting ${user. home}/. m2/settings.xml    [user settings]

Step4: Push your project to Nexus staging repository

Run command mvn deploy in command line or right click pom.xml in Ecilpse IDE, click Run As -> mvn build...

Step5: Release the project in Nexus repository manager

Login your Nexus repository manager account, click close button, Sonatype need to check all requirements rule. Once it done with no errors, release button will be active. Click release to finish the deployment.

Some problems I encountered and how I solved:

  1. When I run mvn deploy , system prompted an 401 error.

Try to reset the user token, and update settings.xml. Run mvn clean deploy again.

2. %JAVA_HOME% is not correctly set

Try to follow this blog How to set JAVA_HOME in Windows 10 to set environment variable JAVA_HOME and run mvn clean deploy again.

3. Javadoc validation and source validation failed

Make sure the plugin of source, javadoc and gpg are called and run properly when you run the command mvn deploy

Last, I hope this blog is helpful for you.

Plan on Release0.4

For release0.4, I will still focus on the project Telescope.

Why do I choose it?

First reason is that I am so interested in the concept of Telescope. In my daily life, I like to read some technique blogs, usually I add the websites to my Favorites folder, and go to these websites regularly to check if there are new blogs or updates. However, it is not efficient and timely. Thus, when I met Telescope, I thought that maybe I could build a similar website for myself to track those blog websites I am following. To do that, I need to figure out how Telescope works and the techniques involved. So, writing code and fixing bugs to Telescope is the best way for me to learn the project.

Second reason is that Telescope front-end is migrating from Gatsby to Next.js, I heard Next.js before and I am seeking an opportunity to learn this react based front-end framework. Now, Telescope gives me this chance to use Next.js, I am ready to start hacking!

Unit test | Continuous Integration

Testing is the critical part in the software development. This week, I added unit test cases and continuous integration on my LinkChecker-CmdLC project.

To better build and manage my project, I spent some time to convert my project to a maven project before I started writing some unit test cases using JUnit.

A maven project has standard directory layout, as well as a pom.xml file managing dependencies and plugins. My project looks like this. All source files are in the src/main/java folder, while test sources are in src/test/java folder. This avoids file path issues.

The hardest part of all test cases was mock a Http Connection, which required to simulate network responses. Mockito was the popular tool to do the mock in Junit, therefore my original try was using Mockito. After failing to implement mock network connection many times, I got help from my classmate. He suggested me to use powerMockito. Here is my final code.

Once you write test cases, the good practice is using a test coverage tool to check your test cases coverage rate.

In addition, I set a continuous integration check for my project using Github Action. It is a very useful tool to manage a project, especially a big and fast moving project with lots of contributors. You can set various test checks base on actions. You can set the checks running on different operate systems, such as windows, ubuntu, etc. I set 6 jobs in the project workflow, that means once a pull request or a push is submitted, github will do these checks for me to ensure the project integration.

Besides Junit, I know that Jest is another popular testing framework. So, I tried to write test cases using Jest to my classmate project. (See pull request)

I know I still need to write more test cases to be proficient in JUnit and Jest, but I am already on the way. I really like this course. 😆

Blockly

The other project I worked on is Google Blockly. Blockly is a web-based visual programming editor. It is a kind of programming game. I know this project because my son’s school is using Scratch to teach coding. Scratch is an educational product which is developed based on google blockly.

Blockly was not a small project, it had a clear and detailed guideline for the developers. I spent times to set up the project, read the documentation, go through the code and find the issues.

Finally I found an issue I thought I could give a try. The issue was about CSS. Comments on the blocks were missing scrollbars. Comments looked like:

The project had a css.js file under the core folder, I tried to find CSS for comments there. But it looks comments CSS was not there. Then I went back to comment.js, and got CSS for comments.

After doing a research on overflow vs text-overflow, I figured out that overflow: hidden could hidden the text, while text-overflow: hidden should hidden the text as well as show a scrollbar.

Therefore, my fix was simple, changing the “overflow” to “text-overflow”. Following is the effect.

The tiny issue does not impact the functionalities, nevertheless, it influences user experience. And sometimes user experience effects customers’ decision. I am happy that my contribution could help this project improve user experience.

PR: https://github.com/google/blockly/pull/4447

How to make searches shareable via URL?

Make searches shareable via URL was the issue I fixed in my release0.3. To implement the feature, my first approach was using browser history, therefore, I did some research on History API.

I used {BrowserRouter, useHistory, useLocation} to record the searching history and get the parameter from URL. The following was the affection at that time.

Since the frontend of Telescope is written by Gatsby, while Gatsby is a React-based open source framework. I suppose that using react-router-dom was compatible. So, I didn’t run npm run build locally before I submitted my first PR. And the fact was the PR could not pass Github integration check, it failed to build.

Then, I took professor’s suggestion to use Gatsby use-query-params plugin.

I changed my code to use {navigate, useQueryParam}.

Later, a new problem appeared, the owner hoped that when user clicked the Back button, the search page could return user’s previous page. Now, it returned letter by letter through URL.

I solved it by using navigate options.replace to replace the current URL in history.

However, options.replace didn’t implement the feature as it should.

At the end, the final PR was modified by my professor, he helped to refactor the structure of Search Page component, remove {navigate} from Search Bar, and set URL updated once the search form was submitted so that the Back button could work the way we want.

Through this practice, I realize that contributing to a big project sometimes is a back-and-forth process, it is not only submitting a PR, passing all the checks and waiting for merge. Most of time, you need to communicate with the project owner, they will give advices on the Pull Request, maybe some improvements, maybe some new related requirements. And the process is a kind of learning and contribution.

Static Analysis – how to use Google-Java-Format and SpotBugs

To maintain an open source project, one important thing is the use of Static Analysis tooling which helps reducing repetitive, tedious, and meaningless code formatting or mini bugs modification tasks.

In working with my LinkChecker project, I use Google-Java-Format and SpotBugs to manage my project complexity.

Google-Java-Format

Google-Java-Format reformats Java source code to comply with Google Java Style, so you don’t need to set the configuration yourself. But if you want to reformat changed lines in a specific patch, you could configure settings in google-java-format-diff.py.

Using the formatter/IDE Integration
  1. Install Google-Java-Format plugin from Eclipse Marketplace, and set it as formatter implementation.

2. Set the formatter to implement once the file is saved.

Using the formatter from the command-line
  1. Download the formatter and run it with:
java -jar /path/to/google-java-format-1.9-all-deps.jar <options> [files...]

In my case, I run the following command:

java -jar lib/google-java-format-1.9-all-deps.jar --replace src/osd/*.java

2. Write a script to run the formatter with “one-step”. My example script is at the end.

SpotBugs

Using the linter/IDE Integration

Install spotBugs from Eclipse Marketplace

Using the SpotBugs GUI
  1. Download a SpotBugs binary distribution zip format .
  2. Extract it into a directory of your choice.
  3. Create a project and run the analysis.

My running result is as follows.

Using the linter from the command-line

I run following command in powershell:

java -jar lib\spotbugs-4.1.4\lib\spotbugs.jar -textui src/osd/*.class

SpotBugs reports the bugs it found. The first Letter ” M”, “H” means the level of the bug. M – median, H – High.

I add “UTF-8” and “StandardCharsets.UTF_8” to fix the bugs.

Writing a script

I write a script to create a simple “one-step” solution for running the formatter and linter on the entire project from the command line

The result:

Using Git Pre-Commit Hook

Last, copy the script in the hooks subdirectory of your .git directory, and name it pre-commit (without any extension). Now, it will be called every time before you make a commit.

PR URL: https://github.com/jossiey/LinkChecker-CmdLC/commit/91505f0025c28128d048018c23d4acce56bac802

Telescope Setup | Gist

From this week to the end of this semester, we will focus on an open source project named Telescope, which is built by Seneca students. I have introduced this project last month.

The first task this week is setting up project environment. The project uses a variety of technologies, such as Redis, GatsbyJS, Docker, Elasticsearch, etc. therefore, it is not easy to get everything done. However, I worked on this project in Hacktoberfest, I have already run the project on my machine. So, I skip this step.

Next, I was required to modified my LinkChecker-CmdLC to check links in Telescope Posts, http://localhost:3000/posts.

This part was easy to implement. I modified the code and run it.

Last, I needed to create a Gist to record the modification and differences between my codes.

There are so many new technologies, I can hardly wait to learn and explore them.