Behind every successful company is a strong tech stack that supports the entire team and user base. For the following NYC tech leaders, the technology that powers their company is everything — and that’s why they carefully selected front-end and back-end tools, like Spark and TypeScript, that best suit the needs of their team and industry. We talked to six engineers who walked us through their strategies for selecting their tech stack. Unsurprisingly, it’s not a one-size-fits-all approach.
Flatiron Health is a platform that connects oncologists, academics, hospitals, life science researchers and regulators to help in the fight against cancer. As such, they need a reliable tech stack that enables engineers to be productive and efficient. Software Engineer Clint Harrison went over the specifics.
What does your tech stack look like these days? What are some of your favorite facets of it?
We use primarily C#/.NET and Python, depending on the team, with front-ends in React and Angular. I work on the team supporting our Python users, and we're working on an exciting migration to modern Python 3 with Bazel, an open source build tool from Google. Our goal is to enable our engineers to be as productive as possible, improve code quality, reduce test times and ultimately ship features faster with standardized tooling.
How do you choose what tech to use to complete a project?
We're continuously starting new projects and products. One of the fastest ways to get up and running is to reuse the infrastructure we already have. Using existing libraries and frameworks — whether that means using already-adopted open-source ones like Flask or internally created frameworks — allows engineers to get started with minimal effort and benefit from standardized logging, security, test setup, etc. As we explore new product areas, Python isn't always the right choice, and we're looking into languages like Go when we have different deployment constraints or need better application performance.
Marketing measurement software company DoubleVerify chose a tech stack that makes it easy for the team to work with large volumes of data without worrying about computations and scalability. Associate Staff Engineer Aravind Ramesh explained what he loves about the technology.
What does your tech stack look like these days? What are some of your favorite facets of it?
Our current tech stack consists of Apache Spark to process data at scale and Apache Kafka to process real-time streams of data. We use Luigi to manage complex batch processing workflow and utilize Hive for data warehousing. Our apps run on Kubernetes, and we use Splunk to capture app logs, infrastructure logs and metrics which help in setting up monitoring and alerts. We operate on the cloud and use Google Cloud Platform.
I love working with Spark, which is a distributed computing framework. Spark makes it easy to process large volumes of data, while the framework takes care of distributing workloads across machines in a cluster automatically. Spark also has very friendly high-level APIs that makes it easier to focus on the data transformations instead of worrying about the computations and scalability. Spark has APIs to process both batch and streams of data, thereby making it a perfect fit for our business needs.
How do you choose what tech to use to complete a project?
Deciding which tech to use for a project starts with defining and scoping the problem and understanding the use cases better. I begin by outlining the goals of the project and make a list of tools and technologies that align with these goals. Any technology that contradicts with must-have goals gets eliminated in the process. Talking to domain experts and teams that have used a specific technology gives me valuable feedback and helps me avoid surprises along the way. I then build a proof of concept to demonstrate the use cases to the team.
Cannabis wholesale ordering platform LeafLink is complex, and as such needs a tech stack that measures up. VP of Engineering Mark Sost explained the strategy behind their selection and what it could mean for the future of the company.
What does your tech stack look like these days? What are some of your favorite facets of it?
LeafLink's tech stack primarily consists of Python (Django), HTML/(S)CSS/Javascript (Vue.js), PostgreSQL, and runs on AWS. We leverage Django's REST Framework to power our APIs and run all of our systems within Docker containers throughout our various environments.
Looking ahead a bit, we're gearing up to move our existing web app to more of a platform centered around service-oriented architecture. This will include breaking out functionality into independently-scalable applications that can be utilized in different ways to create iterative product experiences for our platform. The project will allow us to explore new and useful technologies as requirements grow and change.
How do you choose what tech to use to complete a project?
A number of factors contribute to the technology we use in a project. We look for team familiarity for starters, but also consider if it’s a use case fit (does the tech solve the problem at hand better than an existing tool?); if there’s ecosystem support (will the product be around for a while?); and how complex would onboarding be (how long does it truly take to become proficient in the tech?).
Ultimately, I believe there’s a balance to be struck between allowing your team to explore, experiment and come up with new solutions to complex challenges while also putting guardrails in place so the overall system doesn’t become unmaintainable or unstable over time.
Truth discovery platform Remesh uses AI and human input to provide insight on different demographics. When choosing their tech stack, Lead Software Engineer Dan Reich considers two things: simplicity and usability.
What does your tech stack look like these days? What are some of your favorite facets of it?
We're currently using PostgreSQL and Django on the backend and TypeScript, React, and Apollo on the front-end. My favorite part of our stack is our use of GraphQL and TypeScript. We get to define the shape of the data we need, when we need it — without needing to modify or query our API 100 times. Further, with TypeScript, even though GraphQL doesn't offer the most robust type of safety, we get some assurance that what we get is what we expect all the way from request to click.
How do you choose what tech to use to complete a project?
When choosing a tech stack, typically I do my research, choose the simplest tool for the job, and only scale it up as my needs and requirements change. If my entire team can't use it, it's useless.
Howl is a platform that’s changing the consumer shopping experience by connecting millions of products with the content created about them. The team uses a strategic tech stack to make sure engineers are able to develop, test and deploy code as quickly as possible.
What does your tech stack look like these days? What are some of your favorite facets of it?
We are primarily a Python shop using some parts of the JVM for our larger data pipeline processing and ReactJS on the front-end. We process billions of events every month of our own data using Apache Storm, Kafka and RabbitMQ.
One of my favorite aspects is that our tech stack allows clients to truly see the performance of their commerce content as purchases occur in real-time. Most recently we started using Gremlin and TinkerPop which has given us the freedom to query our database of 80 million products in far more sophisticated ways to provide additional insights to our publishers and merchants.
How do you choose what tech to use to complete a project?
At Howl, we're all about autonomy and empowerment. Engineers are responsible not only for development but also the testing and deployment of their code.
This empowers individuals to research and discover best-in-class tools and platforms for the problem at hand and get them shipped as quickly as possible.
When it comes to choosing technology for big data company Graphika, it’s a team effort. Senior Software Engineer Bridget Keyes explained how everyone needs to be on board with the tech stack in order for it to be most effective.
What does your tech stack look like these days? What are some of your favorite facets of it?
Our core product runs on a three-pillared stack. Our front-end is a React/Redux application, with visualizations built in D3 and Three.js. The front-end is supported by a Ruby API using the Sinatra framework. Behind the scenes, we have several significant data science systems written in Python. This code relies heavily on the pandas and NumPy libraries. Our data storage layer is Postgres and Redis. The whole system is running on AWS, using Ansible and Terraform for configuration management.
How do you choose what tech to use to complete a project?
We’re a small team, and we’re pretty open to trying new things when we need them. When a new project starts and someone proposes a new technical option that they think will work, we will discuss it as a team, weighing the pros and cons and other options to consider. We also loop in our research and analysis teams, who are constantly working on prototypes and specific customer-focused analytic scripts. We’ll put the new technology through its paces in a proof of concept, and then make a final go or no-go decision on using it.
