A fungus may help create gardens to feed long-distance explorers. Danielle Lamberson Philipp

In January, botanist Simon Gilroy and his lab sent tomatoes into space, their sixth plant-based experiment with NASA on the International Space Station. Nicknamed TASTIE — short for Trichoderma Associated Space Tomato Inoculation Experiment — the project aims to give researchers a better understanding of how plants grow without gravity.

“Plants know up from down, right? They don’t have a brain or anything like that, but shoots grow upward, and roots grow downward. They clearly are using directional information,” Gilroy explains.

But remove gravity from the equation and things get wonky. And stressful. That’s where the team is hoping a common fungus, Trichoderma harziannum, will come in. On Earth, the fungus is known to be beneficial for plant growth, making plants hardier in the face of stressors. In space, it may help grow food for long-distance explorers.

“The moon and Mars — where I think we’re going to go — they’re a long way away,” Gilroy says. “You’re going to have to be able to be self-sustaining at a significant level.”

“When you’re muted, people don’t expect these apps to collect data,” says Fawaz (left), with Yang.

Kassem Fawaz’s brother was on a videoconference with the microphone muted when he noticed that the microphone light was still on — indicating, inexplicably, that his microphone was being accessed.

Alarmed, the brother asked Fawaz, an expert in online privacy and an assistant professor of electrical and computer engineering at UW–Madison, to look into the issue.

Fawaz and graduate student Yucheng Yang PhDx’23 investigated whether this “mic-off-light-on” phenomenon was more widespread. They tried out many videoconferencing applications on major operating systems, including iOS, Android, Windows, and Mac, checking to see if the apps still accessed the microphone when it was muted.

“It turns out, in the vast majority of cases, when you mute yourself, these apps do not give up access to the microphone,” says Fawaz. “And that’s a problem. When you’re muted, people don’t expect these apps to collect data.”

Fawaz and Yang, along with colleagues from Loyola University Chicago, used runtime binary analysis tools to trace raw audio in popular videoconferencing applications as the audio traveled from the app to the computer audio driver and then to the network while the app was muted. They found that all the apps they tested occasionally gather raw audio data while mute is activated, with one popular app gathering information and delivering data to its server at the same rate regardless of whether the microphone is muted or not. The findings raise privacy concerns.

“With a camera, you can turn it off or even put your hand over it, and no matter what you do, no one can see you,” says Fawaz. “I don’t think that exists for microphones.”

Turning off a microphone is possible in most device operating systems, but it usually means navigating through several menus. Instead, the team suggests the solution might lie in developing easily accessible software “switches” or even hardware switches that allow users to manually enable and disable their microphones.

Boutilier hopes research like his will nudge automated external defibrillators closer to mainstream implementation. Everdrone

As a kid, Justin Boutilier would get roped into helping his dad, a paramedic and firefighter, perform automated external defibrillator (AED) demonstrations. Two decades later, Boutilier, now a UW assistant professor of industrial and systems engineering, is reimagining how AEDs can save more lives.

Boutilier has detailed the framework for designing a network of AED-outfitted, autonomous flying drones, which could allow the life-saving devices to more quickly reach people suffering cardiac arrest. In out-of-hospital cardiac arrests, survival rates drop by as much as 10 percent for each minute that passes without treatment.

“Ambulances are not fast enough for this, especially in nonurban areas, so drones are just such a good fit,” says Boutilier, whose research harnesses optimization and machine-learning techniques to improve health care quality, access, and delivery. “They’re super fast with straight-line flight. And then AEDs are a relatively light payload, so it suits the drone. The best applications for drones in health care are things that are light and where time is of the essence.”

In January, an off-duty doctor used an AED delivered by an autonomous drone to save a 71-year-old man’s life in Sweden — the first such documented successful rescue. Boutilier hopes that research like his will help nudge the technology closer toward mainstream implementation.

The Badger football team’s high-tech mouthguard will measure details such as speed, impact, direction, force, location, and severity.

Researchers at the UW School of Medicine and Public Health are partnering with the National Football League to study on-field head impacts in order to reduce injuries at the professional and collegiate levels. They will collect data from high-tech mouthguard sensors used by the Badger football team that measure details such as impact speed, direction, force, location, and severity.

Insights gleaned from the data will help decrease head impacts and inform the NFL’s approach to injury reduction. UW football players have the opportunity to opt in to the NFL’s novel program.

“Reducing the risk of sport-related concussions is a priority for athletes, coaches, and health care providers alike,” says Daniel Cobian ’06, PhD’10, assistant professor of orthopedics and rehabilitation and co–principal investigator for the study. “The more we learn about sport-related concussions, the better we can protect our student-athletes from these injuries and maintain a desirable balance between the benefits of sport participation and the risk of injury.”

Data collected from athletes will be anonymized and analyzed by the NFL Players Association’s independent engineering experts and affiliated consultants. UW researchers will also get a statistical analysis that can help the Badger football team advance player health and safety.

The NFL launched the mouthguard sensor program in 2019.

“We are excited to establish this partnership with not only nationally recognized collegiate football programs that have a direct pipeline to the NFL,” says Allen Sills, the NFL’s chief medical officer, “but also with esteemed research universities to expand the universe of inputs from which we can draw conclusions and develop recommendations for how to make the game safer.”

Courtesy of Mezlight

Surgeons spend hours hunched over operating tables, enduring distraction and discomfort under the weight of a headlight as they work to heal their patients. Like any good doctor, UW Health transplant surgeon Josh Mezrich set out to find a solution. In partnership with Madison entrepreneurs Craig ’89, JD’96, MS’04 and Karen Christianson, Mezrich developed MezLight, a light fixture that attaches to the side rail of an operating table and easily adjusts with the help of a flexible arm. According to Mezrich, in addition to being more comfortable than a headlight, the MezLight makes it easier for surgeons to wear face shields, which have become common in the operating room during the pandemic.

The thin, flexible device is self-powered, implantable, and bioresorbable, so once the bone is knitted back together, the components dissolve within the body. Jason Daley

In 2017, Green Bay Packers quarterback Aaron Rodgers broke his collarbone. Typically, it takes about 12 weeks for a collarbone to fully heal, but fans hoped Rodgers might recover early and save a losing season.

So did Xudong Wang, a UW professor of materials science and engineering and an expert in creating thin, movement-powered medical devices. “I started wondering if we could provide a new solution to bring athletes back to the field quicker than ever,” Wang says.

Researchers know that electricity can help speed up bone healing, but “zapping” fractures has never really caught on, since it requires surgically implanting and removing electrodes powered by an external source.

Wang’s latest invention is a major update of that same electrostimulation concept. It didn’t come in time to help the 2017 Packers, but it may help many others by making electrostimulation a much more convenient option to speed up bone healing.

His thin, flexible device is self-powered, implantable, and bioresorbable, so once the bone is knitted back together, the components dissolve within the body.

Initial tests have been successful in animals, and Wang and his collaborators hope to scale up the device so it will work in humans.

The UW telescope gave astronomers their first real peek at polarized ultraviolet starlight aboard the Space Shuttle Columbia. NASA

In 1990, a novel UW–Madison space telescope swept into orbit aboard the Space Shuttle Columbia. It gave astronomers their first real peek at polarized ultraviolet starlight — a form of light that helps us learn about everything from star formation to the interstellar medium, the dusty void between the stars.

Known as WUPPE for Wisconsin Ultraviolet Photo-Polarimeter Experiment, the telescope was one of a trio that made up NASA’s Astro Observatory. Deployed from the shuttle cargo bay during two grueling missions in 1990 and 1995, Astro helped open the ultraviolet universe, something that can’t be done from Earth as our planet’s atmosphere filters ultraviolet light from the stars.

Since 1996, WUPPE has been a prize exhibit at UW Space Place, the university’s astronomy outpost on Madison’s south side. In September, the 800-pound instrument was carefully packed and trucked to the U.S. Space & Rocket Center in Huntsville, Alabama, where an intrepid cadre of volunteers, mostly retired NASA staff, is reuniting the three telescopes in anticipation of Astro’s eventual move to the Smithsonian’s National Air and Space Museum in Washington, DC.

With the delivery of WUPPE, the privately funded project is entering phase two, says Scott Vangen, one of the Astro Restoration Project’s instigators and an alternate payload specialist for the Astro 2 mission. “We’re going to put it together as it flew,” says the retired NASA engineer, whose team has recovered 90 percent of the observatory’s hardware. A few key pieces of the Astro puzzle were retrieved from scrapyards in Florida and Alabama, including the seven-foot-by-seven-foot “cruciform,” a 3,000-pound aluminum frame that cradled the telescopes, and an optical sensor in the shuttle cargo bay.

UW Space Place director Jim Lattis MA’87, PhD’89 feels bittersweet about WUPPE’s departure.

“I have mixed feelings about it, but I don’t have any hesitation,” he says. “Once it belongs to the Smithsonian it will be in the hands of professional curators, and it will be seen by orders of magnitude more people.”

A University Housing employee rattles me awake with his phone, opens my lid, and places the lunch order in my insulated compartment: a grilled chicken sandwich, a blackened chicken breast, chicken wings, and a vegan sloppy joe.

“Very kind,” I say in a friendly computerized voice.

UW–Madison launched me with a whole fleet of Starship Technologies robots in 2019. I was engineered in Estonia and am well trained to handle Wisconsin winters. Rain or shine — or a half-foot of snow — you’ll see me rolling down campus sidewalks without a worry in the world. Except the dreaded crosswalk.

I begin my journey to Library Mall, avoiding a traffic cone and a few gawkers carrying futons. Today, I’m being followed. A writer and a photographer from On Wisconsin are observing me for a celebrity profile. When I accelerate to four miles per hour, they have to break into a jog to keep up.

I make quick adjustments as I encounter obstructions large and small. My front and back wheels help me pivot, balance, ascend, and descend on a moment’s notice. Skateboarders admire my ability to jump curbs, though it does no favors to carbonated beverages in my compartment.

I know my location to the precise inch, with GPS tracking and computer vision. Twelve cameras detect everything around me. Ultrasonic sensors, radars, and neural networks help me sense danger as far as 200 feet away.

As I race down a pedestrian pathway separating the Lakeshore Tennis Courts, I come face-to-face with a navy-blue Toyota that’s gone off-road for move-in purposes. It’s a standoff, paralyzing us both for several seconds. But the car blinks first and readjusts. We pass in peace.

No matter what’s in front of me — a pedestrian, a bicyclist, a dog, or a fellow robot — I stop at a safe distance and negotiate around it. For added safety, my orange flag flaps in the wind, and miniature blinkers signal my intent. I follow pedestrian rules, largely staying on sidewalks.

Once, a fellow robot fell into a manhole on State Street, where construction workers came to its rescue. At the rare times when I do tumble off a curb or get stuck in snow, friendly Badgers usually set me straight. I never said I was perfect. I have a small green scar on my glossy white exterior to prove it.

I’ve reached the hectic corner of Observatory Drive and Charter Street. At intersections, our human developers don’t yet trust us. A handler back at headquarters takes over remotely, guiding me across the street once the 80 bus passes.

It’s sure easy to spot the freshmen today. They’re the ones staring at me with curiosity or amusement. Everyone else dodges me nonchalantly, as if I were just another pedestrian. Which, at this point, I guess I am.

When people block my path, I try to remain patient. “Please let me pass. I’m going to be late,” I say through my loudspeaker. I’m trained to escalate my language based on the level of human interference, and I can sound alarms for emergencies.

I evade dozens of speed-walking students down University Avenue and East Campus Mall, finally reaching Library Mall. I come to an abrupt stop. A group of On Wisconsin staffers stand around awkwardly, waiting for the writer to unlock my compartment with his phone.

“Hello, here’s your delivery,” I say.

He sends me away with a quick swipe to the right.

“Thank you,” I add, starting back to Dejope for my next order.

A robot’s work is never done. •

A handler placing the order in my compartment.

Avoiding obstacles.

Facing off with a Toyota.

The writer extracting the food.

Satisfied customers eating on the Terrace.

Heading off to my next assignment.

Jeff Miller

In the summer of 2021, the UW and Madison Gas and Electric opened a solar panel array at O’Brien Solar Fields in Fitchburg, Wisconsin, giving renewable energy efforts a 20-megawatt boost. There are 60,318 panels in the 140-acre space, and they’re designed to harvest power from the sun for the next 30 years.

“Should your robot butler ask whether you want tea or coffee?” asks Mutlu, professor of computer science, psychology, and industrial engineering. “Or should you merely need to say, ‘Just bring my beverage, robot,’ because maybe you’re going to offload the beverage decision to your coffee machine, so the robot doesn’t worry about it?”

To Mutlu, it sometimes seems he has spent his life thinking about mechanical companions. When he was five years old and his family moved to a new neighborhood in Turkey, children wouldn’t play with him. “I made up a story,” he says. “I said, ‘My uncle brought me a robot to play with, so I don’t need to play with you,’ and here I am at age 42 designing robots.”

His life changed in 1999, when he was 23, working in Istanbul as a product designer for the appliance maker Arçelik. He heard the term smart appliances and recalls, “I realized computers were going to play a bigger role in design. I was totally unprepared to handle that, but I knew being ‘smart’ no longer meant designing dials and buttons. Whatever ‘smart’ was going to be, it wasn’t what I was doing.” So he came to the United States and earned his PhD from Carnegie Mellon in human–computer interaction.

Mutlu develops robotic technologies that seamlessly mesh with people’s lives.

Today Mutlu’s goal is developing robotic technologies that seamlessly mesh with people’s lives. “You need a deep understanding of the context of how robots will be used,” he says. “Otherwise they’re not going to succeed. You can build the most amazing robot, but if people don’t want it, they don’t want it, and it’s not a success.”

With the aid of two postdocs and 12 graduate students, Mutlu holds the reins on 10 projects funded by NASA, the National Institutes of Health, and the National Science Foundation. All have the same goal: to make robots as commonplace as cats, dogs, and delivery trucks. His teams are designing robotic manufacturing tools for Boeing, creating a smart display for the senior-citizen interface Elder Tree, teaching a robot how to perform the surprisingly challenging task of opening an oven door, figuring out how to program robots by talking to them, and field-testing Misty, a robot reading buddy that lives, so to speak, with families and has 24 emotions.

“The People and Robots Lab is a very impressive interdisciplinary space for the study of emerging interactive technologies and one of the top labs internationally for exploring human–robot interaction,” says Selma Sabanovic, director of the R-House Laboratory for Human–Robot Interaction at the University of Indiana–Bloomington.

Just as people learned to take for granted TV, the internet, and voice-based computer systems such as Siri and Alexa, Mutlu predicts that in 20 years robotic technologies will have advanced beyond warehouse and delivery uses to the point where people will buy helper and companion robots as routinely as they now purchase laptops or smartphones.

“This thing will roll into your room, say things, and listen to you,” he says. “Things are about to change in very big ways.”

Mutlu once worked alone. But as a consequence of the 2019 creation of the UW’s School of Computer, Data & Information Sciences and its incredible popularity among students, five assistant professors — Yuhang Zhao, Corey Jackson, Yea-Seul Kim, Adam Rule, and Jacob Thebault-Spieker — are also researching human–computer interaction. Along with faculty at the UW Department of Industrial and Systems Engineering, which is doing similar work on campus, these researchers are developing solutions that enable computer-based technology to improve our daily lives.

Mutlu’s robot Misty is calibrated to boost children’s interest in science.

Each of these new hires spearheads research as diverse as Mutlu’s, tackling everything from medical recordkeeping to household safety to self-driving cars. But one theme unites them — making computers and robots better so they can empower people.

“Now I Can Be a Superman!”

For data scientist Yea-Seul Kim, empowerment might be too mild a word.

“The accurate way of perceiving things has to be a human right,” says Kim, who is developing tools and algorithms to help low-vision people interact with data. “Everyone should be able to make decisions based on the same information. Companies use huge data in clever ways, but individuals have less expertise and fewer resources. My vision is to shrink that gap, so people can make decisions as if they had the techniques to analyze huge amounts of data.”

Kim, Mutlu, and Yuhang Zhao make up the UW Department of Computer Sciences’ Human–Computer Interaction Group. Kim and Zhao teamed up for a study in which they asked visually impaired people how to best replace online graphics with “alt text,” which describes the content of an image. There are nearly 300 million vision-impaired people, according to the World Health Organization, and for them, understanding visual data poses a daunting challenge.

“I want to imitate the way sighted people would skim the visualization,” one test subject told Kim and Zhao. Another interviewee offered a possible solution: combine alt text with an audio version of a graph whose pitch would rise and fall with levels on the graph. The duo’s paper ended up proposing that alt text be automatically created by a system that detects trends in charts’ visual data.

Zhao has another project on the front burner, literally. She, Mutlu, and optometrist Sanbrita Mondal, the head of Low Vision Services in the UW’s Department of Ophthalmology and Visual Sciences, are studying cooking habits of low-vision people.

“These patients can’t function in the kitchen,” says Mondal. “They’re afraid they’ll burn or cut themselves. They eat frozen dinners or reheatable meals, which affects their health.”

Zhao plans to outfit study participants with augmented-reality smart glasses souped up with thermal cameras, depth sensors, and audio alarms. The super spectacles will tell wearers about stove-related dangers using multimodal feedback — colors to depict hot spots, defined contours of objects, and sounds if a wearer’s hand strays close to a high-temperature area. No stranger to assisting people with disabilities, Zhao helped invent the Canetroller while interning at Microsoft. This high-tech “white cane” let visually impaired test subjects navigate a virtual-reality world in a lab. When users tapped into a virtual trash can, carpet, or table, it made warning noises, created vibrotactile feedback, and impeded contact with the object.

“Oh, now I can be a superman!” one trial user told her. The Canetroller may train low-vision people to use standard white canes in safe settings before venturing onto sidewalks, according to Zhao.

“Doing research with people with disabilities has been such a fantastic journey for me, though my background is pure computer science,” she says. “I’ve had many conversations with them to try to better understand their lives. It’s opened up my mind.”

Ending “Note Bloat”

Zhao’s work with patients has a counterpoint in the Information School (or iSchool) and its Collaborative Computing Group, which includes faculty members Adam Rule, Corey Jackson, and Jacob Thebault-Spieker. Rule wants to help stressed doctors. His field is cognitive ergonomics, especially medical informatics — how medical professionals use clinical notes to perform collaborative, data-driven work.

The epidemic of burnout among doctors has been intensified by “note bloat.” Physicians who once scribbled concise patient records now use tablets. U.S. doctors’ notes are four times longer than those of peers in other countries, according to Rule. “The highly trained American physician,” says the journal Annals of Internal Medicine, “has become a data-entry clerk, required to document not only diagnoses, physician orders, and patient visit notes, but increasingly low-value administrative data.”

“I want to develop tools to let doctors leverage the power of data and computation so they can focus more on patients rather than documenting patients,” Rule says. “We can build systems that let them access information in a central place and work more flexibly on the fly.”

Vast Disparities in Access

Kim develops tools and algorithms to help low-vision people interact with data. “The accurate way of perceiving things has to be a human right.”

Human–computer interactions level the playing field. That’s what the iSchool’s Corey Jackson learned during the summer of 2009. A political science major, he had been planning a career in international law when he went on a One Laptop per Child internship after his junior year to the island of São Tomé off West Africa’s coast. He found himself teamed with computer science professors and, ever since then, has focused on human–computer interaction.

Jackson found that children’s homes had no electricity. Classrooms had one outlet, if that. “It made me realize the vast disparities in access to basic infrastructure. The most rewarding part was seeing children’s faces light up when we did programming modules,” he says.

Today Jackson works to transform citizen science at websites like Galaxy Zoo and Zooniverse, where thousands of laypeople collaborate with astronomers on projects with titles like “Help Scientists Search for Gravity Waves — the Elusive Ripples of Spacetime.” No experience is needed to do this crowdsourced research, and the sites offer few guidelines. Newcomers get confused, so Jackson ensures that the technology works and is accessible to users.

“We need to make sure people from different socioeconomic or racial backgrounds can use data and technologies we develop to advance their goals,” he says.

Crisis of Confidence

The same disparity also hurts rural areas. The iSchool’s Jacob Thebault-Spieker, whose expertise is social computing, learned that in middle school when his family moved from Saint Paul, Minnesota, to tiny, impoverished Sebec, Maine.

He grew up a self-described “computer nerd,” but when he saw peers take plush jobs at tech companies, Thebault-Spieker says, “I had a crisis of confidence. Working for Google to make its search engine a few milliseconds faster is important, but it didn’t resonate with me. I started trying to figure out how to use my skills to solve social problems.”

His work focuses on understanding patterns of behavior — geographic or otherwise — that create biases and disparities in systems like OpenStreetMap, Wikipedia, and Uber. He is addressing poor coverage of rural areas by OpenStreetMap, which Tesla uses in its autonomous-vehicle mapping systems. For example, cattle grates deter cows from wandering off pastures but let vehicles pass. Because mappers in urban areas may not know what these grates are, incorrect labeling could cause car accidents.

“If we’re talking about Tesla being the future of self-driving vehicles,” Thebault-Spieker says, “that’s not happening in Sebec.”

“The Robot Doesn’t Judge Me”

While others employ technology to solve specific problems, Mutlu’s robot reading-buddy project explores a more intimate connection between people and machines.

The first incarnation of his study in 2017 used Minnie, a 13-inch-tall desktop robot named after Minerva, the Roman goddess of wisdom. She was placed in test families’ homes for two weeks. The goal? To study whether children — 10 boys and 14 girls ages 10 to 14 — would enjoy reading with her, if that relationship would flourish over time, and if the experience would make them more avid readers.

Minnie had big, black, inviting eyes that moved. To appear more human when speaking, she would look away as if lost in thought. To mimic fidgeting, her white, plastic, humanoid head made random movements. Her stationary body held a microphone and a camera that used facial recognition to locate the child.

Programmed with 25 books that ranged from Goosebumps to Harry Potter, Minnie listened to children read. She appeared thoughtful. She summarized what they said and urged them to do better.

“The biggest shock in our study was that two weeks later,” says Mutlu, “kids were still relating to the character rather than saying, ‘This is stupid. I’m not talking to your robot anymore.’ ” The number of children who thought Minnie had emotions quadrupled during the study. Children said Minnie motivated them to read. They thought she helped them understand and remember books better compared to responses given by robot-less children in a control group. Minnie became a friend. Children were glad she never criticized them. “I liked that the robot doesn’t judge me when I misspell,” one child said. Families sent Mutlu’s research partner photos of Minnie dolled up with clothes. Children made her feel comfy by cushioning her with pillows.

Mutlu is wary of creating machines humans may become too attached to. “Do I want people to fully assume a machine has emotions? I’m skeptical,” he says. “My goal is to make it more intuitive and natural for people to interact with these things. I don’t want to push the boundary of, like, ‘Oh, I want people to have deep relationships with them.’ ”

This past summer Minnie lost her job to Misty, a programmable consumer robot whose base model sells for $2,299. The 14-inch-tall white or black plastic Misty moves on treads. Her arms wave and point. Big, blue, cartoony eyes programmed with 24 emotions dominate her face screen.

Misty’s study gauges whether a robot reading buddy can boost children’s interest in science. Her emotions are calibrated to amuse children into maintaining a heightened interest in such things as mold and spiders. For example, if a scary spider is in a book, Minnie flings up her arms in dismay and retreats. Her eyes turn red, their shapes expressing fear.

Mutlu has young children himself and worries parents might use robots as electronic babysitters. He limits his children’s screen time, and the robot reading-buddy project appealed to him because of its solitary, educational nature.

“I don’t think we’re competing with parents or social interactions, but I completely agree that [unsupervised children using robots] is a risk and something we have to think about,” he says.

For Misty’s next challenge, Mutlu will send her to stay with families for nine months. “What is it going to mean for a social robot to live with a family that long? How is it going to affect family interactions?” he asks. “This is very complicated stuff. We are trying to design it with the best of our knowledge.”

In the future, he says, home robots will be programmed to respond to each family member differently. Such a robot might say to a parent, “I can’t talk to you now” or “Can I play with the child?”

“It will be a very different paradigm for interaction,” says Mutlu.

“Do More, Do Better”

Some fear a future in which killer machines trample humans — and human rights. Mutlu, who is Catholic, is confident robots and other forms of human–computer interaction are part of a greater — and better — plan.

“What are our friends and families for?” he asks. “They’re for helping each other get to heaven. So why not build these machines to help people do more, do better, and do what they’d like to accomplish?”

The question, says Mutlu, is: “Can I get these systems and technologies to support that mission?” •