Contributing to Github Projects

A few months ago F5 released a python-based SDK for the iControl REST interface. I have mostly used iControl REST by building my own functions for working with the BIG-IP and including along with the core logic of my particular task. But the SDK abstracts all the BIG-IP work, allowing the end user to just focus on core programming logic. I’ll be converting some codeshare samples to the SDK in the near future, but I thought I’d start with how to contribute to a github project, namely the f5-common-python SDK. I’ve been working with python for quite a while, but mostly with smallish projects or throwaway scripts. Until now I had not contributed to a large collaborative project. There was a steep learning curve for me with some new technologies I wasn’t greatly familiar with, so I thought I’d share with you in the event you would like to help us on this project, or prepare to help with any of the numerous other projects out there. 

Because I am a caveman, I prefer to look at pictures when trying to understand something so let’s start with that.

The first thing I want to do is to pick a project. In this example, I’ve picked the f5-common-python project in the F5Networks repositories on Github. I need to create a fork of that project on my Github account.

Now that I have a copy of the f5-common-python project in my repository, and I can clone it to my local system.

git clone https://github.com/jasonrahm/f5-common-python.git

(where jasonrahm is my github id.)

Next, I need to link my local copy with the master I forked, so I can keep my local copy updated as other developers work is merged in to the master in the F5Networks repository.

git remote add upstream https://github.com/f5networks/f5-common-python

# Verify

git remote -v
originhttps://github.com/jasonrahm/f5-common-python.git (fetch)
originhttps://github.com/jasonrahm/f5-common-python.git (push)
upstreamhttps://github.com/f5networks/f5-common-python (fetch)
upstreamhttps://github.com/f5networks/f5-common-python (push)

A commit template is helpful for the individuals that will review and merge your changes to instruct them on the changes you are committing. The commit template provided by F5 for this project can be linked for future commits, so I'll do that now.

git config --global commit.template .git-commit-template.txt

At this point I am ready to start contributing! I can research existing open issues, or add a new one I found that needs to be addressed. In my case, I wanted to add support for files, data-groups, and ssl keys & certs. It's important, however, to work on one feature at a time so it's easier for the reviewers to merge and it's less work for you if you did something wrong and then have to go back and fix hundreds of lines of code.

For the first feature, I'll develop support into the SDK for ifiles, so I need to create a new branch from the development branch. First, I checkout the development branch, and then I create a new branch I'll call feature.ltm_ssl_cert_key.

git checkout development
git checkout -b feature.ltm_ssl_cert_key

(where feature.ltm_ssl_cert_key is the name of your feature.foo or bugfix.bar)

Once you checkout your new branch, you can start modifying existing files or adding new files. As you add files, I add them to git as well instead of waiting until right before the commit. I'm using Jetbrains PyCharm for my development, so a lot of this part of the process (the git commands) are handled for me automatically, but to add and commit files, the syntax is:

git add
git commit

(make first line of commit message a meaningful summary)

Before I push my local updates to my fork, I need to do a few things. The first thing is to sync with the upstream repository. We linked it above, now we sync it to make sure what I push forward is not going to cause any conflicts with the master copy.

git fetch upstream
git rebase upstream/development feature.ltm_ssl_cert_key

(where feature.ltm_ssl_cert_key is the branch you want to update before pushing to your fork)

The next thing I want to do is to develop and run test cases. (Depending on your development process, you can add test cases for your feature, or you can build your features out of your test cases. YMMV on whether this step exists here or a couple steps prior.) In any event, this project will not accept pull requests without unit and functional tests, so I can't proceed until these are written. Once I have my tests written (using the pytest module,) I can run my tests. I typically run them as I write them to make sure they are passing.

py.test --bigip 172.16.44.15 --username admin --password admin f5/bigip/tm/sys/test/test_file.py
==================== test session starts =======================================
platform darwin -- Python 2.7.10, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
rootdir: /Users/rahm/PycharmProjects/f5-common-python, inifile:
collected 5 items

f5/bigip/tm/sys/test/test_file.py .....

===================== 5 passed in 0.07 seconds =================================

The final step before I push my changes to my fork is to run the style checker. The coding style is strictly validated to make sure lines aren't too long, spacing in between functions and classes are appropriate, imports are done properly, etc. PyCharm gives me pop up warnings automatically as I code for the style issues, but you can run

flake8 --exclude=docs .

as well (and I do this step afterward for the extra verification away.)

Finally, after development, rebasing, testing, and style checking is done, I can push the code to my fork.

git push origin feature.ltm_ssl_cert_key

(where feature.ltm_ssl_cert_key is the branch name you want to push to your github fork)

Now moving back to Github, I need to create a pull request to begin the merge process. I select my branch and then click “New pull request."

In the next screen, I see the base fork and base as the F5Networks/f5-common-python repository and development branch, and the head fork is my fork and compare is the feature branch I developed.

This will land on the F5Networks side, with each of my commits listed so the reviewers can see what I changed incrementally. Also, I see that the pull request kicks off some automated testing. Passing tests locally on my development box doesn't necessarily mean a clean test run remotely. The usual culprits are differences in TMOS versions I didn't test against, missed flake8 errors, or just simply a faulty test run. I can close and re-open the pull request to have the tests run again if I think the latter issue to be the case.

After testing is complete and passing, I can add a note to the pull request that the feature is complete and testing and should be reviewed for merge. And congratulations to me, I've contributed to my first "real" project on Github!

Conclusion

There’s learning to code in a language, and then there’s learning how to contribute to a larger collaborative sustainable effort. I’m learning a lot and I’m thankful for the small team of people at F5 who are making great things happen with python (among other things) and taking on me as a mentoring project! Special thanks to Tim Rupp, Za Wilgustus, and Wojciech Wypior for their insights and patience as I dug in.