Microsoft on the road to open sourcing the search system used to power Bing

Kareem Anderson

Once again, Microsoft’s new development spirit has the company going to the open source well for inspiration and foresight. According to InfoWorld, Microsoft is in the process of extending “its open source activity to include projects derived from work done on its Bing Search engine.”

So far it seems that teams within Microsoft have been able to jump start three major projects based on the BitFunnel full-text search system powering the company’s search engine and intelligent mind behind Cortana, Bing. The first of the projects can be found on the GitHub site for BitFunnel titled BitFunnel text search/retrieval system. Unfortunately, the project seems to be in a nascent state as of right now with very little code organization and no documentation accompanying the README file it thus far.

The second of the three projects is classified under WorkBench, which appears to be a tool for translating text for use in BitFunnel. The last of the three projects and perhaps the most in-depth exploration from Microsoft comes in as NativeJIT. Referencing InfoWorld’s analysis, it would appear the Microsoft’s NativeJIT project will be able to take expressions using C data structures and repurpose them into “highly optimized assembly code.”

The idea behind the NativeJIT project stems from optimizing conditions for developers when:

  • The expression isn’t known in advance.
  • The cost of compiling the statement to assembly will be more than offset by the number of times the expression will be run.
  • “Latency and throughput demands require a low cost for compilation,” meaning there are so many queries being processed in this manner that compilation can’t afford to be a bottleneck.

The overall development of Microsoft’s newly open sourced endeavors could mean the company is looking into having Bing become a much wider networking web for new technologies. Open development of Bing could also mean that newer standards and features are adopted faster resulting in a tool that can generate and process search engine front ends while producing just-in-time machine learning results.

Microsoft’s BitFunnel projects are in early development and have ways to go, but it should be encouraging for Bing, Windows, Windows 10 Mobile, IoT, HoloLens and Xbox fans to see the company looking beyond its walls for future tech.