Wednesday, December 4, 2019

Benchmarking on More Platforms

In the previous article I looked at the performance of different mathematical operations and found that the Decimal operations of C# take around 10 times the time it takes for same Double.  I also found that the performance profiles of different processors vary greatly.  So, I decided to take a look at several different processors and see what other interesting things I could find.

To start with, I ran the code on my Intel Compute Stick to see how the Atom processor performed.  It actually put in a solid and relatively flat performance similar to the Core i7 we looked at last time.  Here are the addition results:
And the results for multiplication operations were:

Note that multiplication beats addition yet again.  I believe I know why, but will save the explanation for later when I dig even deeper into the underlying code that is being generated.  As a hint, take a look at the multiplication tests, I believe it is an artifact of the test rather than an actual instruction speed difference.

To get another architecture I am going to run out and buy a new computer I need to get a bit creative.  I am an Azure head, so looking at the processors available on the Virtual Machines I noticed that the Lsv2-Series run on the AMD EPYC™ 7551 processor which would be interesting.  So, I will create an L8s-v2 in the East US 2 region.  I ssh'd in and used the information from my Installing .NET Core article to install .NET Core and sftp'd in the code.  I ran the test, downloaded the results and deleted the VM (a $464.26 a month burn rate is more than I want to mess around with).  The results were...interesting.  The addition results were:

And the multiplication results were:

The multiplication results were right in line with the addition results, and the decimal was actually longer!  That is the first time the results came out like that, so we need to dig in to figure out what is going on.

Monday, December 2, 2019

All Benchmarks Lie - Or Why Benchmarking Matters

I just finished taking a C# class given by a peer at Magenic who just happens to be a Microsoft MVP for C#. :-)  I love working with smart people!  I has been at least 3-4 years since I spent any time actively learning about C#.  I have picked up other topics (like Pivotal Cloud Foundry) and languages (like Flutter) but I haven't worked on the main tool I use every day for work.  For Shame!  Ralph and I spent a bit over a year doing a Code Dojo with other developers, but that was back in 2013.

As part of the course, the question came up about why anyone would use Double when Decimal has better precision.  We talked through it in class, but I thought it would be illustrative to actually run a benchmark on sample code and show the differences.  So, I ran up a quick benchmark test and proved that addition and multiplication of Decimals are around 10 times more costly than equivalent Double operations.  My laptop results for addition were:

The results for multiplication were:

It was strange for me to see that multiplication was faster than addition, but the magnitudes were not surprising.

As the capstone for the course we had to do a project using C#, so I decided to expand on the idea of benchmarking the different operations by running the tests on different architectures.  Another peer is writing a series of blog posts on running Kubernetes on a Raspberry Pi cluster and I happened to have a Pi 3 A+ that I am using for another project so it was an obvious first choice.  The results were similar, but somewhat less consistent.  The decimal operations cost about 10 times that of the others, but the  byte multiplication operations are almost twice as fast as the long integer multiplication operations.  There were also a greater difference between the multiplication operations and the addition operations.  Here are the addition results:

And here are the results for multiplication:

I have an older Dell PowerEdge R710 that I picked up from TechMikeNY last year to play with some Cloud Foundry stuff so I decided to see what the Xeon processors looked like running the same code.  This is more like what would be expected to be seen in a production environment.  The addition results look like:

And the multiplication results were:

Wow!  Those results looked significantly different than the previous ones.  For one thing, these are the first results that I had seen with the MultiModalDistribution warnings, so I ran them again to make sure the results were consistent.  The results were similar, but not the same.  They were more similar to each other than to the other architectures.  Here are the results for the addition tests:
And here are the results for the multiplication tests:

I went on to do tests on a couple of different Azure VMs, but I am going to save that is for a different article. :-)  I also have plans to dig into these differences a little more and see if I can't track down the underlying causes for the displayed differences.

Saturday, November 30, 2019

Installing .NET Core (quick hit)

I am working on a series of articles and need to install .NET Core 3.0 on various Linux platforms.  This is just a quick article so that I can find these links later:

  1. Centos, Redhat, Fedora, Debian, Ubuntu, SUSE, or OpenSUSE:
    https://docs.microsoft.com/en-us/dotnet/core/install/linux-package-manager-centos7
  2. Raspbian:
    https://edi.wang/post/2019/9/29/setup-net-core-30-runtime-and-sdk-on-raspberry-pi-4
  3. And finally, I need to get Mono installed in the same as well as Windows:
    https://www.mono-project.com/download/stable/

Friday, November 29, 2019

Another look at benchmarks

A little over two and a half years ago, I have a blog post investigating the difference in the memory footprint between the use of StringBuilder and general string concatenation.  The version of BenchmarkDotNet that I used was v0.10.2 and I used Framework 4.6.  Now is a good time to retry using the latest in both, v0.12.0 and .NET Core 3.0.  So, let's start a new project based on Console App (.NET Core) and name it as before:
New project with Console App (.NET Core) selected


Then let's add the BenchmarkDotNet NuGet package:
BenchmarkDotNet in NuGet

And accept the changes:

Add the source from the previous article and give it a run by changing into your directory and run via dotnet StringsVSStringBuilder in an administrative command prompt:

When it runs, you will notice that the first thing that it does is compile the benchmark.  That is different than what the previous version did, but it isn't really surprising.  It makes sense to compile the application into a known state, and the tooling has come a long way.

Looking at the results, we find that we need to do a bit more to get the memory output that we are looking for in this instance:

The MemoryDiagnoser is no longer enabled by default:

Let's add it via an attribute on the test class:

Now, running the tests again and we get the memory output again:

As before, the concat method allocated a little more than twice the memory of that of the StringBuilder version.  It is also a small, but significant bit faster.

Sunday, June 30, 2019

Super Sunday Tech Roundup #5


On the road this week for PI Planning for my current client and walking around in the Mall Of America I had a chance to visit the Lego Store and the MindStorms EV3 looks pretty amazing!  The software is as impressive as the hardware, making it easy to transition between complexity levels as a child grows in understanding.
MindStorms EV3
https://www.lego.com/en-us/mindstorms/about-ev3
I also am getting ready to put Schroeder on a plane for Texas and loaded his Kindle PaperWhite with books using Calibre.  Whenever I need to use it I remember what an awesome piece of software it is.  It just works, and does an amazing job.

Calibre ebook management
https://calibre-ebook.com/


Sunday, June 23, 2019

Super Sunday Tech Roundup #4

I am getting more and more into Kubernetes, and want to do some cluster work on my desk with multiple machines and Dell 710 servers are just too heavy to carry around.  So, I started looking into smaller machines and thought about Raspberry Pi’s and how to work with a stack of them and found the PicoCluster site and fell in love.  I know what I want for my next birthday!

Less is More
https://www.picocluster.com/

As part of my rabbit hole search above, I was talking with a colleague and he said ‘I have these 2 Android phones, could you use them?’ And after some searching, I found the answer was no but we can do some cool stuff on it.

How to Turn an Android Device Into a Web Server
https://www.makeuseof.com/tag/turn-android-device-web-server/

Sunday, June 16, 2019

Super Sunday Tech Roundup #3

I am not a Git Guru, but I found Write yourself a Git and interesting article.  I am inspired to build out a similar tutorial in C# but  I doubt I will take the time.  Too many other things drawing my attention and limited time.
Write yourself a Git
https://wyag.thb.lt/
Last year I got one of the Intel Compute Sticks to play around with.  My initial goal was to use it to front playing games or to do a media player at home.  But it just hung from the back of a monitor at work for 6 months gathering dust.  But now!  I am going to install Fedora Linux on it and make it my Kubernetes Master for a portable cluster.  I should get an article or two out of that.
Intel Compute Stick
https://www.intel.com/content/www/us/en/products/boards-kits/compute-stick.html