Microsoft prides itself on running on its own code, for the most part, and likes to point out successes when the company’s latest technologies make its products run better. Such is the case with Bing, Microsoft’s cloud service search engine that runs on “thousands of servers spanning many datacenters across the globe.” In a post on their blog today, the .Net engineering team announced that Bing is now running fully on .Net Core, the slimmed down and cross platform version of Microsoft’s .Net managed common language runtime.
Bing has always run on the .Net Framework, but has been able to move to .Net Core 2.1 after some recent API additions. The move has produced some impressive results, clocking a 34% improvement in internal server latency over previous versions. This has been accomplished with the following changes:
Vectorization of string.Equals (@jkotas) & string.IndexOf/LastIndexOf (@eerhardt)
Whichever way you slice it, HTML rendering and manipulation are string-heavy workloads. String comparisons and indexing operations are major components of that. Vectorization of these operations is the single biggest contributor to the performance improvement we’ve measured.Devirtualization Support for EqualityComparer<T>.Default (@AndyAyersMS)
One of our major software components is a heavy user of Dictionary<int/long, V>, which indirectly benefits from the intrinsic recognition work that was done in the JIT to make Dictionary<K, V> amenable to that optimization (@benaadams)Software Write Watch for Concurrent GC (@Maoni0 and @kouvel)
This led to reduction in CPU usage in our application. Prior to .NET Core 2.1, the write-watch on Windows x64 (and on the .NET Framework) was implemented using Windows APIs that had a different performance trade-off. This new implementation relies on a JIT Write Barrier, which intuitively increases the cost of a reference store, but that cost is amortized and not noticed in our workload. This improvement is now also available on the .NET Framework via May 2018 Security and Quality RollupMethods with calli are now inline-able (@AndyAyersMS and @mjsabby)
We use ldftn + calli in lieu of delegates (which incur an object allocation) in performance-critical pieces of our code where there is a need to call a managed method indirectly. This change allowed method bodies with a calli instruction to be eligible for inlining. Our dependency injection framework generates such methods.Improve performance of string.IndexOfAny for 2 & 3 char searches (@bbowyersmyth)
A common operation in a front-end stack is search for ‘:’, ‘/’, ‘/’ in a string to delimit portions of a URL. This special-casing improvement was beneficial throughout the codebase.
All of that adds up to a faster and easier to manage Bing. Deployment of new versions is much faster, with the latest worldwide update happening only 2 days after the release of .Net 2.1.
The Bing engineering team is working closely with the folks at .Net to make both Bing and .Net faster and better.
