Cracking the Case: How 3 Software Engineers Solved Their Biggest Technical Challenges

In 2014, an unmanned Antares rocket bearing a Cygnus capsule and 5,000 tons of cargo lifted off over Wallops Island, Virginia. The mission was to resupply the International Space Station in low Earth orbit. Yet seconds after launch and just as the vehicle cleared the tower, the turbopump suffered an explosion, and the Antares rocket fell back to Earth. No one was hurt, but the questions were immediate: what happened, why did it happen and how could the issue be quickly fixed? For the engineers, it was time to solve the challenge and get the next Antares rocket back in the air again. One year later, they did.

For software engineers in the tech world, launching a new product can feel a lot like taking a rocket into the upper atmosphere, not to mention smoothly docking it with other existing technologies. The challenges with product launches and updates can be enormous: deadlines, conflicting tech solutions and the pressure to meet customer expectations on the front end loom large. Most engineering successes are earned through untold hours of time, teamwork and effort. 

We sat down with engineers at three New York tech companies to discuss their biggest technical challenges, how they were solved and what lessons were learned in the process. We discovered that by applying perseverance, teamwork and innovation to the issue at hand, almost any challenge is solvable. Just as important, the growth achieved can be immeasurable.

 

At Unqork, a DIY enterprise SaaS platform, Senior Engineer Jennifer Mak came across a major challenge when her DataGrid’s scrolling feature completely altered the underlying HTML from one table element to two, and cut off the cells’ text editing state. In the face of a host of new obstacles, Mak and her team ingeniously looked to the cell editor in their own Position component, which injects a portal at the bottom of the page. The hard work and eventual solution helped her grow on a personal and professional level.

 

What’s the biggest technical challenge you’ve faced recently in your work? 

I’m a senior software engineer on the platform UI team where we specialize in front-end development. Our team is building a high-quality design system that will ease the developer experience, increase productivity and give the UI a cohesive look and feel. We knew that there would be a heavy focus on our DataGrid component since it’s used in multiple feature areas of the Unqork UI. A DataGrid is a data table that’s packed with features for data manipulation and rendering — including draggable rows, grouped columns and row virtualization, to name a few. We decided to use AG Grid as a starting point for our DataGrid component, since it comes with many of our feature requirements out of the box. Another plus was that other teams at Unqork were already using AG Grid, so they were familiar with its API. 

Of course, it’s never as easy as you would hope. We quickly realized when our design strayed even slightly from what AG Grid provided, implementation became very difficult. This was especially the case when it came to adding scrolling capabilities to the DataGrid. Enabling AG Grid’s scrolling feature completely changed the underlying HTML structure of the grid from one table element to two — one pinned to the left and one that scrolls on the right — so all the CSS that we had set previously for one table had to be reworked. 

When the major cosmetic issues were fixed, unforeseen issues began popping up, which made it very difficult to gauge the amount of work remaining. The negative margins that we set to achieve the desired look for our focused cells were getting cut off by overflow properties that allowed scrolling to work. This minor deviation in the design was something we were willing to compromise on. However, the scrolling feature also cut off the cells’ text editing state, which was a major blocker since users would not be able to see the entire cell as they entered information. And, of course, when the implementation was finished and merging was in sight, dozens of unit tests needed updating and our visual regression tool was pushed over the edge by the number of states that required testing. 

 

How did you and your team overcome this challenge in the end? 

I attribute the success of this feature to my persistence, focus and our team dynamic. Everyone brings a positive energy and encourages each other to be patient and persistent in problem-solving. This allows for creativity and thinking outside the box. There were times when I had run out of solutions and turned to the entire team for new ideas of how we could potentially get to the solution. Taking the time to think things through, presenting options to the team and incorporating feedback was invaluable. 

Ultimately, we ended up wrapping the cell editor in our own Position component to implement a text editor cell that scrolls with the rest of the DataGrid. AG Grid has a built-in portal option for the text editor but it doesn’t scroll with the rest of the table, leading to the detriment of our user experience. I could not easily access the actual HTML object of the internal AG Grid portal so customizing the internal component was not a viable option. Our own Position component injects a Portal element at the bottom of the page, affording us the flexibility to achieve our desired functionality. Success arrived from persistently tackling each challenge as it came in a supportive and creative team environment. 

 

Taking the time to think things through, presenting options to the team and incorporating feedback was invaluable.”

 

How did this technical challenge help you grow as an engineer?

I have learned and grown so much as an engineer from tackling issues like DataGrid scrolling as we have built out the Unqork design system from the ground up. Working on a design system requires you to be very detail-oriented and patient — it also involves a lot of troubleshooting and accounting for multiple use cases. Patience and open communication are skills that I constantly hone, professionally and personally. At first thought, picking up a ticket in September 2020 that wasn’t merged into our codebase until January 2021 sounds stressful. No developer wants to be perceived as incapable of completing a task. Having an environment where the quality of the work is valued and persistence, patience and creative thinking are appreciated is a great feeling. I have learned to not be afraid of “rollover” and to stay vocal and optimistic about my work.

System integration and smooth loading times are incredibly important at Rokt, a popular platform for e-commerce technology. Director of Engineering DJ Seo understood that drop rates could negatively impact revenue, so he sought out innovative solutions on both the application and client side. The result was exciting and affirming at the same time.

 

What’s the biggest technical challenge you've faced recently in your work? 

Performance matters. A case study into Rokt partner’s drop rates has shown that this could negatively impact revenue by seven percent. 

Rokt’s integration is mixed amongst multiple technologies competing for time and resources on a web page at load time. From the first DNS request, we are downloading assets from CDN, calling APIs, initializing the environment to show the Rokt solution, sending data to support placement and offer selection, identifying customers, retrieving placement/offer configuration and finally rendering to screen. This is a lot to do at a time when speed is absolutely critical.

In order to achieve our performance goals, we needed to conduct a top to bottom assessment of our stack and the related technologies. We had to consider front-end optimization, infrastructure and network latency along with various other factors. Additionally, the performance is not only affected by us but also the clients’ systems and their third-party components all combined. Hence, the performance improvement was driven by the need for “on the fly” technology innovations both on the application and client sides, while simultaneously raising the bar for our infrastructure technology at its core.

 

How did you and your team overcome this challenge in the end? 

We employed deeper monitoring technologies to measure the performance and availability of our products first. We believed that we could preload resources so they would be readily available when required. We initially came up with the concepts of “cold” and “warm” starts for preloading resources. The changes we have made for the “cold start” preload allow Rokt code to parallelize loading of required scripts and static assets earlier on the same page. Meanwhile for the “warm start” we attempt to do as much preparatory work as possible by loading assets on pages well in advance so that we do less at the critical point of downloading resources, executing them and displaying Rokt solutions on client web pages. 

Furthermore, we also introduced distributed tracing to analyze and optimize the communication between microservices. Simultaneously, we set up various content delivery service providers with real-world settings, compared their performances to figure out which service provider catered best to the varied end-users of Rokt partners and integrated well with Rokt back-end system architecture. We then employed the best provider.

In three months we reduced our overall loading time by 30 percent!

 

How did this technical challenge help you grow as an engineer?

I still remember my web programming 101 lecture. In the lecture, I was asked to execute a simple telnet command to make a GET request to a server. My engineering journey started from there. 

We went back to the very basics and started reconnecting the dots. Everything was fair game and up for debate. Every time we uncovered something new, it added to my understanding of the problem as a whole. I delved into many areas of the web stack throughout this project, especially front-end optimization, networking and content delivery.

 

Jason Meredith is a lead DevOps engineer at Caesars Digital, a platform for betting and gaming. Because the site requires a service hosted by a third party, different platforms such as iOS and Chrome have different interpretations of the cross-origin resource sharing (CORS) policy. The solution to the problem began with AWS CloudFront, and ended with a firm belief that Jason could tackle any challenge, and always learn more in the process. 

 

What’s the biggest technical challenge you’​​​​​ve faced recently in your work? 

Our site requires a service hosted by a third party. Sadly iOS, Android apps and the Chrome browser for the desktop all have different interpretations of how loose or strict the CORS policy has to be. So much so that even though it’s the same site they require different header responses depending on which app is viewing it. Additionally, our app on one or two of these platforms required additional headers to add to the frustration. Our third-party service doesn't have the capabilities to address these issues.

 

How did you and your team overcome this challenge? 

To mitigate the issue we utilized AWS CloudFront, setting up a cache behavior to trigger a Viewer Response Lambda, while having our third-party vendor's URL be set as the CloudFront Distributions Origin. Then in the Lambda, we wrote custom Python to transform the data getting returned to the viewer. This allowed us to have absolute control over how the headers are returned to our site's viewers.

 

Make one big problem into a handful of smaller problems and then start solving those one by one.”

 

How did this technical challenge help you grow as an engineer?

Working on infrastructure most of the time, I think DevOps gets obfuscated from working directly with client-facing applications. This gave me a sneak peek at the challenges our mobile team faces as well as a look into mobile: DevOps. This also gave me a lot more respect for the capabilities of AWS. They had all the puzzle pieces ready for me to put together. Then all I had to do was roll up my sleeves and a little Python to bring the whole project home. 

The hardest part of the project was collecting all the information, including the variables I needed to solve the problem and test the acceptable responses. Once I got all the necessary information and validated some theories, it only took a day of work to solve and implement everything.

This is one of those problems that felt like when it was presented to me it felt insurmountable at the beginning as there were too many unknowns. That being said, I had to follow a philosophy I had been following for the longest: Make one big problem into a handful of smaller problems and then start solving those one by one. It helped reinforce the idea and feeling that I am good enough technically, I just need to remember I’m also always learning.