Microsoft Bing posted a look back on the launch of the new Bing AI search and chat features, documenting both the good and bad of this new search experience in a blog post named The new Bing & Edge – Learning from our first week.

The good. Overall, most searchers seem happy with the results, Microsoft said. In fact, 71% of searches graded the results a “thumbs up” in the search interface. “increased engagement across traditional search results and with the new features like summarized answers, the new chat experience, and the content creation tools,” Microsoft added.

The not so good. Microsoft added that it has been a learning experience and that learning, with search, “can’t be done solely in the lab,” but needs to be done in the real world to see how the real world interacts with it. And yes, Microsoft has found out a lot in just a week since it launched.

• Timely information can be a challenge for the new Bing search. Microsoft wrote, “we are finding our share of challenges with answers that need very timely data like live sports scores.” Even though the Bing chatGPT integration is much faster and more timely, as Brodie Clark found, Microsoft said it is not always fast enough. As a result, Microsoft is “planning to 4x increase the grounding data we send to the model,” they wrote. And also, Microsoft might add a “toggle that gives you more control on the precision vs creativity of the answer to tailor to your query.” Our early test drives showed some early issues as well.
• Chatting for fun has been an area that Bing did not necessarily anticipate would happen so much. Microsoft said the noticed chats for “general discovery of the world, and for social entertainment.” This is something they “didn’t fully envision” for the chat feature but searchers are using it in this way. Some chats go 15 or more questions and some have gone on for over two hours, again, not something Microsoft expected. Some may be leading to multiple personality disorders with Bing. To resolve or adapt, Microsoft said:
  • “Very long chat sessions can confuse the model on what questions it is answering and thus we think we may need to add a tool so you can more easily refresh the context or start from scratch”
  • “The model at times tries to respond or reflect in the tone in which it is being asked to provide responses that can lead to a style we didn’t intend.This is a non-trivial scenario that requires a lot of prompting so most of you won’t run into it, but we are looking at how to give you more fine-tuned control.”
• Technical issues. There have been numerous technical issues and bugs, many have been addressed already and some are still popping up but Microsoft is working on them. Issues such as “slow loading, broken links, or incorrect formatting,” they said. Personally, I was invited to use the new Bing a week ago Tuesday, when it first launched, but for some technical reason, the interface refuses to load for me – Microsoft is working on the issue, I am told.
• New features. And of course, Microsoft continues to work on new features such as booking flights, sending email, share searches or share answers.

Why we care. Again, watching this form of search evolve in real-time has been fascinating. As marketers, you need to follow where your users and consumers go to find the information you might be able to answer. Learn to quickly adapt to see where your links show in these new search experiences and see how you can leverage these interfaces to gain traffic. It might be too soon, but following the space is not just educational and useful but also, at this point in time, entertaining.