The first time I met Jon Friedman, corporate vice president of design and research at Microsoft, was in 2019 when he was spearheading a new open approach to design at Microsoft to make its hardware and software look like it came from one company. That’s a largely successful work in progress, so these days Friedman has, like a lot of Microsoft employees, been focused on AI.
I spoke to Friedman last week, ahead of the redesigned Microsoft 365 Copilot app announcement, to get a better understanding of the impact AI is having on human designers and creators.
“The change is quite dramatic, and the opportunity is quite dramatic,” says Friedman. “The job changed for us. The entire industry and Microsoft were trying to figure out what we can do with this new material that helps adapt computing to people, and it turns out you can do a lot and you can do a lot fast.”
In recent months, Friedman’s own job has changed because of AI, so he’s focused on the content as well as its design. ”Suddenly the design job is how do you edit,” says Friedman. “Even my job over the past 6–8 months has become an editor-in-chief job of the product, not just the design leader.”
As Microsoft attempts to build personalized AI agents or AI-powered apps, the design system has to adapt to the very real challenges of meeting this vision, and the tools to build that system are constantly changing, too. Whether it’s Adobe apps or Figma, AI features are increasingly being built into creative tools that push designers into an era of editing and using AI as a tool, rather than having to create everything from scratch.
The fear and uncertainty around how AI will impact human creativity has resulted in a very real backlash against the use of generative AI in advertising, movies, video games, and elsewhere. While it was initially obvious when an artist or designer used AI to create art, the tools are advancing so quickly now that it’s becoming far more difficult to spot “AI slop.”
Microsoft’s visual design team has been experimenting with using generative AI tools as part of their production process since 2022, and the team revealed this week that it recently made a Surface ad with generative AI. Designers created several top-down stills of a Surface device being used in different locations, and then fed them into generative AI tools like Hailuo and Kling to add motion for the video.
The result is a 55-second ad for the business versions of the Surface Laptop and Surface Pro, which has been available on YouTube for two months without anyone in the comments spotting Microsoft’s use of generative AI. Microsoft says it turned to AI for the ad due to “an immensely tight deadline alongside various resource and budget constraints.”
Friedman, like many working in tech, believes that AI will increasingly be used like this, as another tool for creators and designers instead of something that will simply replace them. He used DALL-E to illustrate his son’s book about managing childhood anxiety last year, and recently used a variety of AI tools to edit videos and create a website.
”As a maker, I loved it. I have this intent, and I don’t have all the capabilities, but I have a lot of the capabilities, and sometimes, I want to sit there and move pixels around, and other times, I want to get a big idea out,” says Friedman. “I think there’s this thing that’s going to happen in the world where disciplines are going to meld together in new ways. A creative maker’s strength is not just going to be in their specific skill set, but their ability to think big and creatively and then take that creative thinking and drive towards better outcomes in the world.”
Friedman thinks that engineers will eventually have their own AI designer and designers will have their own AI engineer, so that these two disciplines will overlap in different ways.
I’ve been experimenting with a variety of AI tools and models for months, and while I’m a relatively competent writer, I’ve found that AI tools can often help in the early stages of a project or a pitch. I don’t use AI tools to help me write articles, but that’s something I’m skilled at, so I don’t need AI for help there. I have used AI tools to help me edit my personal photos and videos though, because I’m not fully confident at that.
I do feel a sense of guilt using AI tools, though. I’m still unsure exactly where I stand in the use of generative AI debate. As a technology enthusiast, it’s exciting to see what AI can unlock, but as a journalist, I’m as nervous about the future of AI as artists and designers are. I’m unsure whether my job will even exist in 10 years. As a young child I also witnessed my father’s career disappear when the print presses rapidly moved to digital. Back then, you either had to re-skill and learn digital imaging tools like Photoshop or get left behind.
“I think what we’re going to see is resistance at first because people fear what it means for them on some level,” says Friedman. “I think there’s a little bit of it’s hard to get over the hump of what it means for you, but then once you see it as another tool in your toolbox, it’s really powerful, for both the things you love to do and the things you may not love to do. For me, it supercharged my makership.”
I’m hopeful AI will supercharge human creativity, but at the same time Microsoft is preparing for a world of automated AI agents that it dreams of being “digital colleagues.” The line between AI as a tool and AI as a replacement for skilled human jobs looks like it’s about to be tested, leaving us all to wonder what comes next.
- Microsoft 365 Copilot redesigned with new search, image, and notebook features. Microsoft is launching a newly designed version of its Microsoft 365 Copilot app soon. The app has mainly been used by businesses as a hub for accessing Office documents, but Microsoft has been gradually pushing people to use its AI tools over the past year. The Microsoft 365 Copilot app will soon have an updated AI-powered search, a new Create feature for generating images using OpenAI’s GPT‑4o model, and Notebooks that operate like miniature AI projects.
- Bethesda officially reveals Oblivion Remastered, available now. The worst kept secret in gaming was finally made official this week: Oblivion Remastered. After developer Virtuos leaked images of the game earlier this month, The Elder Scrolls IV: Oblivion Remastered got shadowdropped on Tuesday for Xbox Series X / S, PS5, and PC. It’s already proving popular on the Steam charts, as fans of The Elder Scrolls series jump in to play a remastered version of the nearly 20-year-old title.
- When the world connected on Skype. Microsoft is shutting down its Skype service next month, after more than two decades. Rest of World has put together a list of stories from readers about how Skype helped them stay connected to friends and family. Skype really did help change the way we communicate.
- Microsoft’s Xbox app is now available on LG smart TVs. If you want to play Xbox games on your LG TV, but you don’t own an Xbox console, there’s now an app for that. The Xbox TV app is now rolling out to select LG smart TV models, including LG’s 2022 OLED TVs, select 2023 smart TVs, and newer models and smart monitors that are running webOS24 or higher. You’ll need an Xbox Game Pass Ultimate subscription, a compatible LG TV, and a Bluetooth controller to start streaming Xbox games to your TV.
- You can now run Windows 11 on an iPad if you’re in the EU. The Digital Markets Act (DMA) has opened the door to allow emulation apps to run on iPadOS, and YouTuber NTDev has demonstrated how you can now run things like Windows 11 on an M2 iPad Air.
- Windows for the Game Boy, with a playable version of Minesweeper. RubenRetro has created a version of Windows for Nintendo’s Game Boy. It even has an MS Paint app that works with the Game Boy Printer, a playable version of Minesweeper, and a built-in media player. I love it.
- Microsoft Copilot can now ‘see’ what’s on your screen in Edge. Microsoft is making Copilot Vision, the AI feature that can interpret what’s on your screen, available free to use within the Edge browser. You can ask Copilot Vision to compare items you’re looking at on a webpage, help with recipes, or even use it for cover letter brainstorming. The more powerful mobile version of Copilot Vision still requires a Copilot Pro subscription, but this screen viewing experience is now available free of charge.
- Xbox will soon let you purchase games in its mobile app. Xbox is finally rolling out the ability to purchase games and DLC, as well as sign up for Game Pass directly from its mobile app. To be able to offer this, Microsoft has had to remove remote play within the Xbox mobile app, in order to comply with app store restrictions. Microsoft originally wanted to offer both the ability to buy a game and stream it immediately inside its Xbox mobile app for Android, thanks to a court ruling. But Google appealed the ruling, leaving Microsoft to have to adapt its Xbox mobile app while it waits for a final decision.
- Xbox has some new Doom Slayer accessories. We’re just a few weeks away from Doom: The Dark Ages, and Microsoft is preparing for the launch of this new Xbox game with not one, but two special edition Xbox controllers. You can pick up a regular Xbox controller with the Doom Slayer’s armor for $79.99, or go for a transparent red top Xbox Elite 2 controller for $199.99. There’s even a $54.99 Xbox Series X wrap, too.
- Microsoft provides a progress report on its security initiative. Security has been the number-one priority for nearly a year now at Microsoft, and the company is now providing an update on its progress with the Secure Future Initiative (SFI). Microsoft is now using a “Secure by Design UX Toolkit” that has been tested by 20 product teams and rolled out to 22,000 employees to help build more secure products. Microsoft also says its latest policies, investigation methods, and behavioral-based detection models have prevented $4 billion in fraud attempts.
- New Copilot Plus PC features are now being tested. With Recall’s imminent launch, Microsoft is now turning its attention to improving Click to Do on Windows 11 Copilot Plus PCs. Two new text actions are currently being tested, with a reading coach that will help you practice reading out loud and an immersive reader that creates a distraction-free zone around any text you select on your screen.
- How secure is Microsoft’s Recall feature? Security researcher Kevin Beaumont has been trying out the version of Recall that’s about to be released, and he’s found that it’s greatly improved this time around, but there are still some concerns. Microsoft says Recall requires a fingerprint or facial recognition to use it, but Beaumont found you can open Recall using the four-digit PIN unlock option with Windows Hello. The sensitive data filtering also doesn’t appear to work reliably, which is something I’ve witnessed myself.
I’m always keen to hear from readers, so please drop a comment here, or you can reach me at notepad@.com if you want to discuss anything else. If you’ve heard about any of Microsoft’s secret projects, you can reach me via email at notepad@.com or speak to me confidentially on the Signal messaging app, where I’m tomwarren.01. I’m also tomwarren on Telegram, if you’d prefer to chat there.
Thanks for subscribing to Notepad.
