��������

In UTSA’s virtual environments laboratory, tables are covered with mobile phones, headsets, and game consoles that intermingle with projectors, big screen TVs, and cutting-edge technology in virtual and mixed reality. Taking in the entire view, it might seem chaotic to some. But for computer science professor John Quarles, who heads up the lab, and the students who use it, this place is an experimenter’s paradise. It’s where researchers are taking consumer products used for entertainment and transforming them into tools for teaching, rehabilitating, and solving complicated problems.

“This is the time for using VR in research,” Quarles says. “Right now is when VR is huge, and it’s going to keep getting bigger. There are lots of companies and lots of money behind it.”

The tech media proclaimed that 2016 would be the year of virtual reality, and UTSA researchers exemplify what that could mean beyond the average consumer product or video game. With multidisciplinary applications—that can train new heart surgeons, create games for kids to learn how to be safe on computers, and create virtual teachers for kids with autism—UTSA has its finger on the pulse of advancing the technology of virtual and augmented realities.

Virtual reality has been around in some form since the 1950s and came to a bit more prominence in the 1990s. Ford Motor Co. has been using VR for manufacturing since 2000. Yet the biggest boon to the technology was more recent—Facebook’s $2 billion purchase in 2014 of the then-obscure VR-development company Oculus Rift. Since then, the industry has exploded. Google, Microsoft, Samsung, HTC, Amazon, and Apple have all also entered the race in developing reality-based technologies. Tech experts predict future mobile phones will all be VR-ready. Even amusement parks are getting into the fray. This summer San Antonio’s Six Flags Fiesta Texas added virtual reality to its Superman roller coaster, allowing riders to wear a headset and become immersed in the world of Metropolis.

All of this means an expected boom in the job market with employers looking to fill areas that already have an abundance of vacancies. While jobs are mostly in development and engineering, they do cross over into marketing and communications, security, sales, and business development as well as the arts, music, and graphic design. At the university level, by combining VR and other digital realities—plus the expansiveness of big data and data visualization—a world of opportunities opens up for disciplines from the arts to history and anthropology.

“We are early adopters of technology at UTSA,” says College of Engineering professor Yusheng Feng. “Even before Facebook bought Oculus Rift we recognized the importance of that technology. How we stay competitive is to develop applications for using those technologies, whether in research or for the general public.”

To keep the technology up to date, UTSA has upgraded VizLab, the Advanced Visualization Lab, and its centerpiece VizWall. The lab is part of the Center for Simulation, Visualization, and Real-Time Prediction, established by Feng in 2009. In addition to the cutting-edge facility, the College of Business has established a new master’s of science in data visualization program, which begins this fall [see sidebar “Talk to Me, Big Data”]. The goal is to help faculty and students across the university unlock the potential for use of big data, high-resolution imaging, and virtual reality in their own research.

Virtual World, Real Solutions

Punching sharks in the nose may not immediately come to mind as a therapy method for multiple sclerosis patients. But it does when that patient is UTSA’s Quarles. In 2007, he was diagnosed with the disease, which his sister also has. Once he started physical therapy a couple of years later he stumbled upon a realization: “It just wasn’t any fun. I thought to myself, I could do better than this.”

Quarles, who loves to experiment and play video games, says he is always stressing the message to his two daughters that science and research is great fun. (He also bemoans the lack of women in technology and engineering fields—computer science, in particular.) So as a personal project he developed Shark Punch, a VR method of aquatic physical therapy. For the past year Quarles has been working on the game, which involves strapping a waterproof smartphone to a user’s chest. A second phone attached to a dive mask places the user in a virtual underwater world filled with sharks. As part of therapy, the user then strikes out to punch a too-close shark on the nose. “The idea is to think outside of the box when it comes to virtual reality gaming,” he says. “Games can have a positive effect on people’s lives, and that’s what we’re striving for.”

That message is seen throughout the San Antonio Virtual Environments laboratory at UTSA, where Quarles and his students are developing programs to help underserved populations. Quarles believes the explosion in VR popularity, especially with gaming, will raise questions about accessibility, so the team is trying to address that earlier rather than later. “I’ve been doing a lot on rehabilitation with persons with disabilities,” he says. “It’s not always a quality-of-life issue but just people with disabilities who want to play VR games too.”

At any given time, there are multiple ongoing projects. In one of those, being conducted in partnership with a Brazilian doctor, a student is creating a physical therapy game that instructs the user to walk through a house and do everyday chores. Patients can watch their own progress on screen as they try to open medicine bottles, open a door for a pizza delivery, and answer a phone [see sidebar “Taking On Virtual Solutions”].

“Virtual reality can revolutionize how people live their daily lives,” says Gayani Samaraweera M.S. ’14 Ph.D. ’15, who has moved to Seattle to work as a software engineer at Amazon. “We work with understudied special-user groups, like people with mobility issues. The sense of impact is quite unique.”

Creative Collaboration

Standing over a patient, scalpel in hand, a heart surgeon’s split-second decisions can mean the difference between life and death. With that in mind, a collaboration among faculty and students at UTSA and The University of Texas Health Science Center at San Antonio was born. “We created the bypass surgery simulator not really for skills training but more to train in making real-time decisions,” Feng says. “We wanted to be able to psychologically prepare them for when they are in surgery, so if we can simulate that experience, it would provide another training tool.” [Watch a video of the bypass simulator in action.]

The project was one of the first to be funded by the San Antonio Life Sciences Institute, which provides annual grants to partnering teams from UTSA and UTHSC. Feng, Quarles, and their students worked with Edward Sako, chief of UTHSC’s adult cardiac surgery division, to come up with the simulations. “Members of the team observed an operation and saw what we were dealing with and what it looked like,” Sako explains. “The idea was to look at situations that don’t occur very often in real life. And while it’s good they don’t occur often, that also means our trainees may not see that type of situation, making it harder to train for situations that occur in their career.”

The simulator exemplifies ways virtual reality can cross disciplines and research projects. Other collaborations include students working with Quarles and computer science professor Greg White to develop a game for kids about cybersecurity and a recent Ph.D. graduate who reached out to the College of Education and Human Development to research how VR could assist children with autism. Chao Mei ’16 was interested in virtual reality when he started as a Ph.D. student in computer science following receiving his master’s from UTSA in 2015. After seeing autism in the news he started to consider the possibilities. “I was thinking that, maybe, if they are not willing to talk to the people in real life, they could have an imaginary friend or an ideal person they would want to talk to. That’s something we can make in the virtual world.”

He got in touch with interdisciplinary learning professor Lee Mason, the executive director for the university’s Teacher Education Autism Model Center, an applied behavior analysis clinic that serves the needs of children with autism and other intellectual disabilities in South Texas. Mason supplied the expertise about children, learning, and autism that Mei needed for his research—which later became his doctoral dissertation. “This was novel, groundbreaking stuff,” Mason says of the research. “He did see a positive difference in how kids with autism reacted to the virtual world and the virtual teacher, but we need more research on how it can be applied in real life to help kids communicate with others.”

And as more and more children at younger ages are using computers, the need to teach them ways to stay safe online increases. Ph.D. student Brita Munsinger and master’s student Bushra Zahed have teamed up to create a fun way to do just that. “Nowadays, kids are exposed to the internet, online media, and sometimes cyberbullying,” Zahed says. “We want them to have ideas on what to do in these situations—how to take care of their privacy, how to protect themselves from cyberbullying, and how to protect their password.” It will be designed to be used as a regular computer game, but it could venture into the virtual world. Instead of looking at the room, the user would be inside the room.

This isn’t just about a different platform for the sake of variety, experts say. “You can explain to people what virtual reality is, and they think they understand it. But you don’t really get it until you experience it,” Quarles says. “When you experience it, that’s when you have a stronger physiological and emotional response to it.”

Ancient Arts, New Directions

Virtual painting, dance performances generated from body-tracking software, 3D modeling, and moving images across a 24-monitor, high-resolution screen. Mark McCoin, art professor and coordinator of UTSA’s new media program, knows the endless possibilities that pop up by combining the seemingly opposite disciplines of computer science and fine art. “We focus on the moving image, sound art, and digital art,” he says. “What makes new media art different is we focus on interdisciplinary, time-based works using technology. If we could get into virtual reality—using a tilt brush or 3D aesthetic installations where you go in and create these whole new worlds—that is an artistic experience. There are so many ways we can play around with this.”

McCoin sees that potential being unlocked in UTSA’s VizLab. Along with the high-resolution VizWall, the lab contains a haptic feedback device (users can feel the pressure of touching a virtual object as if they were touching a real one) for surgery or injection simulations, a 3D stereoscopic monitor, and a holographic visualization device, among other technology.

In addition, staff and faculty at UTSA’s Center for Archaeological Research see a future where prehistoric rock shelters that they’ve already documented in 3D could be made virtually available to the public. “It’s important for communities to be involved with their past so that they understand what happened before them,” says Katherine Smyth, data systems coordinator for the center. “We can harbor all of that data, but we need to have creative and engaging ways to share that with others. The technology offers new, exciting ways to do that.”

Realizing the need to help researchers like Smyth and McCoin use the lab to its full potential, the university funded full-time staffers for the lab as well as upgraded equipment and made the supercomputer that runs everything more powerful. That means an even greater ability to process information using big data research.

Just one example of UTSA’s use of big data is a new federally funded study of chronic medical conditions. Mechanical engineering professor Adel Alaeddini is beginning with 12 years’ worth of data from more than 100,000 patients being monitored for up to 32 different chronic conditions, including hypertension, depression, lower-back pain, and post-traumatic stress disorder. “The main object is identifying the major patterns in the disease,” he says. “We can tailor this approach to each patient so that their physician knows that if this person isn’t treated within the next few months, they will probably develop a certain chronic condition.”

Whether scientists who are researching biomedicine or artists who are creating new pieces, the goal is to make access and use of the lab easier, including by adding workshops, scheduling tours, and increasing awareness, says Harry Millwater, associate dean for research in the College of Engineering. “We want this to become the face of research for UTSA,” he says. “Our measure of success for the lab isn’t just going to be what research is getting done but how many colleges on campus are using it.”

Meeting Real-world Demand

College of Business alumnus Roberto Viejo ’15 has an inside look at how the VR job market is growing. He’s a partner in local virtual reality company Level 2 VR, which will have a game released on all major platforms this year. Viejo says he’s pleased his alma mater is focusing on these types of computing resources and sees even more areas where future expansion could be possible, such as expanding computer science programs to tackle gaming, which brings together all sorts of talent, like graphic designers, animators, and developers.

“As far as virtual reality and augmented reality, we are all in the learning-to-walk stage,” Viejo says. “But this is the future, and these are the jobs of the future.”

As the cybersecurity and VR industries continue to grow, Quarles believes even more opportunities exist for synergy among disciplines, like art and business, especially in the gaming industry. To that end, he and Greg White are hoping to cultivate partnerships across UTSA’s colleges. Quarles says he would love for the university to expand its offerings by starting a game development program. “The artists are the creative ones, so if we had someone focused on teaching 3D modeling or animation, which falls between arts and sciences, that would help our students become multidisciplinary team players. I would love to work with more artists.”

According to industry publication Road to VR, the full impact of the demand for the job market may even be years away; the market is visibly continuing to grow with more than 200 companies in 2015 hiring for VR. Experts predict that the jobs will continue to expand into the medical, military, and education fields. Professional sports and even news agencies have also jumped into the VR field. The New York Times, for example, launched a VR app to be used with Google Cardboard and now produces immersive stories regularly.

But there’s potential for other progress as well. “With all of the different applications, VR even has the potential to break down racial, gender, and other societal biases via manipulating perceptions, providing different virtual experiences,” says UTSA alumna Samaraweera. “The potential of these technologies is just incredible.”