Hello everyone. Continuous Integration (CI) penetrated the world of software development long time ago and for many developers it’s an integral part of a software development process, which allows you to create better code while maintaining the convenience of your development. Previously, setting up CI required significant efforts and money, but now it has become much more accessible, easier and even free. If you are interested in how to configure CI for your .NET Core open-source project, please read the article below.

Disclaimer

Despite the fact that the main language of the demo project is F#, the manual is fully compatible with the projects written in C#.

Table of contents

  1. Introduction and necessary tools
  2. Creating a repository
  3. New project
  4. Build
  5. Test
  6. Deploy

1. Introduction and necessary tools

The article uses a demonstration repository. We have created tags for each chapter in the repository, a complete list of which can be found at the end of the article.

If you want to repeat what is written in the article you will need:

2. Creating a repository

Tag: InitialMaster

Let’s get started. First of all, we will need some sort of version control system. We will use Git and GitHub.

Create a new repository on GitHub, select a template for the .gitignore file and the project license. .gitignore file is needed in order to avoid unnecessary files in your repository (for example, the results of the build process). The license is needed so that you can understand the scope of the allowed actions with your code. For this project, we chose the VisualStudio template and the MIT license.

After creating the repository, we need to clone it – take the full copy and put it on the local machine. To do this, install Git locally and execute the following command:

git clone PATH_TO_YOUR_REPOSITORY

Now you have a local repository associated with the remote repository on the GitHub. Keep in mind, Git has good documentation, so if you stumble upon something – go here.

3. New project

Tag: NewFSharpProject

Now, having a repository, we are ready to create our project. Navigate to the folder with the repository and run the command to create a new project.

dotnet new web -lang F#

Important – in order this command works, you need to install the .NET Core SDK. If you are running Windows, you will most likely need to restart the OS in order for the system to pick up new paths for dotnet.exe. And if you are not ready for F# yet (and this is a very funny language, I recommend), you can skip passing the lang parameter and then dotnet will create a web application for you in C#.

After completing this command, you will receive three new files:

  • Startup.fs – here is the configuration of our server
  • Program.fs – here is the server start point
  • FSharpWebAppWithCIDemo.fsproj – and this is the description of the project structure

Despite the minimalism of this, it is enough to launch the “hello world”.

Now let’s agree on the structure of the project. Each project must be in its own folder, and the root folder will contain only configuration files, regardless of the projects.

In the end, we should have something like this structure (further, I’ll use Visual Studio to make changes to the solution):

– FSharpWebAppWithCIDemo // main project folder

– – BLL // project with basic logic

– – WebServer // web server and it’s settings.

Important! In F#, the order of files and projects is important. Files that are higher in the project tree do not have access to files that are down the tree. This means that our web server will be able to access any of the projects (it is at the very bottom), but the project with business logic will have access only to its own code. On the one hand, this is somewhat unusual, on the other hand, it immediately structures the code. Why, for example, the database should know about a web server, right? In general, the less one module knows about other modules, the less likely it is to break it when working on some other module. In other words – low coupling high cohision.

And one more, still an important point, about Linux systems. As you know, projects written in .NET Core are cross-platform, and can be executed on Linux systems. But our project file (.sln file) is not fully cross-platform. Therefore, if you work under Windows OS, the paths to projects in .sln file will not be written correctly – using the backslash character: “\”. Unfortunately, this is a problem now. And it will be fixed soon. But if during a build, you see the following error on the server or on your local machine:

 error MSB4025: The project file could not be loaded. Could not find file ... MySuperProject.fsproj.metaproj.

Most likely this means that you need to go into the .sln file and manually change the “\” to “/”.

Now your project is ready. Check that it can be successfully built and let’s move on.

4. Build

Tag: TravisSupport

We already have a working web application on F#. So let’s start configuring our CI process and let’s start it from the simplest – setting up the build.

Let’s create a new branch and call it TravisSupport. By the way, if you are developing a new feature, it’s good to do it in a separate branch, not in the master. First, you will be independent of your colleagues and their commits. Second, in your local branch you can make any commits without fear that they will fall into the main branch and break anything. And third, if you make a merge with the flag –squash then all your commits will merge into one and will look like if there were not 1000 commit attempts. Important thing – do not forget to remove the source branch, otherwise, you can get a conflict.

But, closer to the point. For CI, we need a tool that will collect our project, test it and do what we ask. As such an instrument, I chose Travis-CI. It is free for open source projects, relatively simple and we have some basic experience with it.

To integrate with Travis we need:

  • Give Travis access to our repository.
  • “Explain him” what is required from him.

The first point is quite simple. Open https://travis-ci.org/ and log in with your GitHub account. Then add your repository to the list of available repositories. If you do not see your repository, use the SyncAccount button.

The second point is much more interesting. The management of the CI travis process is performed using the .travis.yml file. Here is the documentation. Copy this file to the root of the project and let’s take a look at what it contains.

language: csharp
dotnet: 2.1.4
sudo: false
env:
    global:
        - DOTNET_CLI_TELEMETRY_OPTOUT: 1
script:
    - dotnet build FSharpWebAppWithCIDemo.sln -c Release
  • language – the language on which the application is written. Travis does not distinguish between C# and F#, and there is no fsharp directive. But this file is perfectly suitable for projects in C#.
  • dotnet – we can immediately specify the version of dotnet to be sure about the build.
  • env – this section is responsible for environment variables. As you can see, we added DOTNET_CLI_TELEMETRY_OPTOUT there to turn off the telemetry transmission in the build process. Any variable can be submitted here, and it will be passed by an argument to our build.
  • script – and this section is responsible for what we want from our travis. In our case, we just want to start the build in the release mode for our application.

This is the simplest and most basic CI scenario. In simple words, with each pull request we try to build our application. If the build is successful – all good. If no – we’ll see a warning right in GitHub and via email. The settings for running the CI process are easily configured both in the .travis.yml file and in the project settings on the site. And if you are interested in additional features of Travis (for example, the choice of OS), then you should go here.

5. Test

Tag: CodeCov

So now it’s time for testing. Generally, we have already violated the principles of TDD, without writing tests before writing the code. Let’s fix it.

Under the Bll project, create a new BllTest project and add BllTest.fs. In the same project install xUnit.
Let me remind you that the structure of our project should now look like this:

— FSharpWebAppWithCIDemo

— — Bll

— — BllTest

— — WebServer

In the Bll project, create a new bll.fs file with the Say module and the hello function (we have broke again the TDD by writing the code before the tests)

namespace Bll
module Say =
    open System
    let hello() =
        "Hello user #" + Guid.NewGuid().ToString()

Now test it, namely that the result of calling this function will always be different from the previous one.

In fact, neither the code itself nor the test itself is important for us. The only important thing is their ability to integrate correctly with Travis and CodeCov. Run the test manually, make sure it passes and let’s continue. After we have the first test, we need Travis to run this test. For this, in .travis.yml file you will need to add one more line to the script section

- dotnet test -c Release --no-build BllTest/BllTest.fsproj

Here everything is obvious – we run tests without build (since the build artifacts are already obtained in the previous step) saving our time.

But this is not enough. We need code coverage statistics. To do this, we will need a coverlet.msbuild tool that perfectly integrates with msbuild and creates reports in different formats, including the opencover format we need. Open the Nuget tab and install this package for the BllTest project. In other projects, it is not needed. After that, change the previous line in .travis.yml to this:

 - dotnet test -c Release --no-build BllTest/BllTest.fsproj /p:CollectCoverage=true /p:CoverletOutputFormat=opencover

So, we have tests, they can run and we even have some kind of report on the code coverage. But this is still not enough. Now we need something that can process this report and show its results. Here we can use Codecov which is also free for OpenSource projects. Log in to Codecov using your GitHub account and add the required project. Finally, update .travis.yml by adding a new section after_script.

    after_script:
        - bash <(curl -s https://codecov.io/bash)

Now that all the preparations have been made, we can do the comit and get our deserved badge with 100% coverage.

6. Deploy

Tag: DeployToAzure

Relax and get some coffee, the end of infrastructure set up is very close. Let’s begin preparations for our deployment process.

In short, deployment is the process of delivering the results of our work (build artefacts) for execution. It can be done manually or automatically. We’ll show you how to set up a demo in Azure. If this does not work for you, you can use the build artifacts that Travis will prepare for you. It is enough to change dotnet build to dotnet publish and send the results to your server, for example:

  • AWS S3
  • GITHUB PAGES
  • NPM
  • HEROKU
  • GOOGLE APP ENGINE

First, we need to create our own web application. This is very simple. Go to Azure, then Create a resource -> Web App or just click this direct link. There we create the name of our future site, the subscription and OS where our server will be deployed. Since we use .Net Core, you can choose either Linux or Windows. Create your website, check that it really works and let’s start setting up the deployment. To do this, add two files to the project’s root folder:

  • .deployment // describes what we need to call for the deployment
  • build.cmd // describes what to do during the deployment

At first glance, build.cmd looks pretty scary, but inside there are quite trivial things:

  • The presence of Node.js is checked // you will not believe it, but for deployment process Microsoft uses Node.js
  • Variables with paths are exposed // where our files and server files will be stored
  • kudusync installation// utility to copy artifacts to the server folder
  • dotnet publish command is being run // compiles our code
  • Publish results are copied to the wwwroot folder. // return the result to the server to run our application

After we added these files, we only need to link our web application to the repository on GitHub. To do this, select our application, then DeploymentOptions -> Github -> Project -> Branch. By the way, you can also set up and load tests there. That’s all, now we need to wait until everything is synchronized and deployed.

Tags

  • InitialMaster
  • NewFSharpProject
  • TravisSupport
  • CodeCov
  • DeployToAzure

References

Useful links:

Tags: