Software Engineers in Test at Google – Covering your (Code)Bases
DIEGO: Hey, everyone. My name is Diego. Today, we’re hosting at Hangout on air on the Google Students page about our Software Engineer in Test position. I’m here with an awesome group of SETs, which is how we call the Software Engineer in Tests.
To talk about what we do and how it impacts our products that you use every day. For those of you who don’t know, SET means Software Engineer in Test. We’re a special type of engineer inside Google with very exciting challenges.
So let’s get started by introducing each one of the SETs that we have right here. First off is Karin. KARIN LUNDBERG: Hi. So my name is Karin Lundberg. I’m the automation lead for Chrome on Mobile.
So that’s Chrome on iOS and Android. I’ve been at Google for about five years. And before Google, I worked as a developer for Novell Identity Manager. And I worked on development tools. DIEGO: Nice.
The next one is Johnathan. JONATHAN VELASQUEZ: Hi. My name is Jonathan Velasquez. I’ve been at Google for a little over five years. And I’ve worked on different products, including Gmail, Orkut, Google Sites.
And now I work on Google+. DIEGO: Chaitali. CHAITALI NARLA: Hey. I’m an SET on the Google+ back end, responsible for providing recommendations. I’ve been at Google three years now. But I joined as a recent grad after completing my Masters from University of Florida.
So hi to all you Gators out there. DIEGO: And last, but not least, Dave. DAVID CHEN: Hi. My name is Dave Chen. I’ve been at Google for about four years now. And I’m a Tech Lead Manager SET leading a team of 12 people on ads, specifically dynamic ads and mobile display ads.
So I got my Bachelor’s Degree at Ohio State. Go Bucks. And then I got my Master’s Degree from Johns Hopkins. Prior to Google, I was at Yahoo for two years as a Senior Software Engineer. And before that, I was at Lockheed Martin working on R&D.
DIEGO: All right. I’m going to pick on Dave now to get started with the basics. So I’m pretty sure people are really interested in knowing what a Software Engineer Test at Google does. So can you tell us a little bit about it? DAVID CHEN: Sure.
I’d be happy to. You know, to me, it means really two things. We focus on developer productivity, one, and product quality, two. Right? So if you think about the traditional testing role, it’s nothing like that.
Everybody at Google is responsible for testing. Development at Google is just a little bit unique. There are no architects. There’s no testers. Everybody’s responsible for design, implementation, testing.
What SETs specifically work on are automation, tooling, best practices, culture shift for our development process to focus on the two areas I mentioned, developer productivity and product quality. DIEGO: All right.
And I think Chaitali also has opinions about it. CHAITALI NARLA: Sure. So I see us actually as people who improve the engineering productivity of the team. And that means building the frameworks that can run tests reliably and fast.
Given how fast we work at Google for shipping great products, I think that’s a real big challenge. And it also means making tools to help your team debug better, write great code. And this is for all Google engineers, including us.
DIEGO: Thanks, guys. This is very important for people to know. But I know that, in the tech industry and probably for many of the people who are watching this, test is highly associated with quality assurance, QA.
Jonathan, can you tell us if that’s true. JONATHAN VELASQUEZ: No, it’s not. And it’s not too closely related to testing either, as they have said. Actually, when I hear this misconception, I often try to think of this with an analogy to what happens in the car industry.
Sometimes you see these commercials of the brand new cars going into a lab. And then you see all these machines going and making the car go crazy. You see all these robots accelerating the car, making it brake, making sure that all of the systems work.
Sometimes these machines are, maybe, even more complex than the actual car that they are testing. Well, something very similar happens here. If we think of a product such as Gmail to be the car, and then an SET would be the person that actually designs all these robots with those complex sensors.
We make sure that we actually have the tools and we bring up this infrastructure to actually go ahead and test a particular product, or maybe several different products. DIEGO: Thanks. But in both roles, a Software Engineer and a Software Engineer in Test are building cool stuff.
What is the difference between them? JONATHAN VELASQUEZ: OK. So there is a slight little difference, in terms of the actual work where both engineers are writing software. So we both go through the same process.
Whenever we want to write a new framework, we have to go through a design. We do code reviews. We also have bugs assigned to ourselves that we need to fix. So in that sense, it’s pretty similar. I think the key difference is that our software is assigned to test, whereas the software of SWE might be assigned to send an email or create a post.
I think the cool thing about an SET compared to a SWE though is that our work may impact different products at the same time. For instance, if a particular testing framework is useful to Google+, then why not use it in Search or use it in Gmail? DIEGO: All right.
And let’s hear from Karin, who hasn’t talked until now. KARIN LUNDBERG: So I agree with a lot of what Jonathan said about the differences between a SWE and SET role. I often actually say, when I explain the role, that SETs are really SWEs.
We just focus, as Jonathan said, on testing. And one of the other differences is we focus often on the product as a whole. So we are working on making the entire product good. So on Chrome Mobile, I’m working on making all of Chrome great on mobile.
Whereas, a lot of SWEs work on specific features. So they might work on streams or something specific in G+ or work on some feature of Chrome, whereas we have to make sure the entire works. And our testing frameworks has to work for the entire product.
So that’s really some of the key differences. We write code, as Jonathan said. We write at least as complex codes as the SWEs. And in some cases, I’ve worked on teams where we actually have been changing the code written by SWEs and make it better.
So that’s also some interesting things. And I just also want to say that you might think that the SWEs are like the most important things for a product. That’s not true. SETs are really critical.
We have very few manual testing. Very little manual testing is done. And on Chrome, we ship every week. On G+ and others, they ship even more frequent than that. And we could not do that, if we didn’t have the SETs put in great testing frameworks and set everything up so everybody can have the tests.
DIEGO: So given those differences, Chaitali, can you tell us a little bit about the interaction between the SETs and the SWEs, in general, like doing projects, doing meetings, doing decisions? CHAITALI NARLA: Absolutely.
And that’s a great question. So the interaction between SETs and SWEs is that we sit physically together. So we see a lot of each other. We also get feature requests from SWEs, because they are our clients.
They are the people we are making these frameworks and tools for, including ourselves, but primarily for them. We also act as consultants for the SWEs. So right from the design phase, we get involved.
We’re doing design docs. We also do code reviews for them and generally give input on how this product can be better designed so that it’s testable from the very beginning. Conversely, they also do design reviews for us.
They do code reviews for us, because we are implementing these giant frameworks and tools for them. So our code, like Karin mentioned, is at least as complex as them. And that does require all the steps that a good software product requires.
DIEGO: Jonathan, what about on your project? How is the interaction in between the SWEs and the SETs? JONATHAN VELASQUEZ: It’s a lot like Chaitali said. We work very closely together. We do code reviews to each other.
Occasionally, when we develop a new framework, we’ll get a lot of questions from them. We’ll get feature requests. I think what’s important to emphasize is that there is no real like what happens in other companies where there is a sort of a competition between the testing team and the development team.
This is not at all like that. This is more of a very collaborative atmosphere. We both work towards the same goal, which is to deliver high-quality products to our users. DIEGO: So now let’s try to get people even more excited.
And let’s have Karin tell us a little bit about the projects that SETs work on. KARIN LUNDBERG: So I’ve been working five years at Google and worked with a lot of SETs on different projects. We do a lot of different things.
I have worked with developers. I have sat with developers and helped them write better codes, understand how to write unit tests. Not all developers have tried that before, so working very closely with them.
Education is a big thing in that way. Actually sitting with the developers and writing the code with them and showing them how to do things. And education has been mentioned previously. I’ve also implemented different test frameworks for web applications– Google Sites, G+, different kind of ways, so we can test it, written the frameworks.
And then developers and SETs and others have collaborated in actually writing the tests. I’ve also worked on tools. And I’ve worked with a lot of SETs that have worked on tools. Specifically, I’ve worked on a tool in my 20% time where I worked on a tool that measured how testable certain Java code was and would kind of tell you, hey, this is where you need to look at changing your code so you can actually write better tests for it.
And then, I joined Chrome mobile a little more than a year ago and really been focusing on Android specifically on actually improving the testability. We had a lot of tests, but not all of them were actually passing, or they were passing some of the time.
So improving that so we could test for a better measurement of the quality of the product. And for iOS, we’ve been implementing test frameworks for casting stability logs and making sure that we don’t actually crash and all that stuff, once we ship to the app store and get out to users.
So a lot of different products all over the thing. DIEGO: Yeah. I can tell. And I know that Dave actually works in Ads. So can you tell us a little bit about the challenges that normal SETs in Ads face? DAVID CHEN: Sure.
Yeah. Ads is an incredibly complex system. I think we’ve got something like a few thousand engineers working on Ads. And it’s kind of this massive system. And you think of it kind of like as, maybe, a 747 or just a complex kind of aircraft that’s flying in midair.
And we’re making changes to it all the time in midair. And everything has to work, right? You can’t break something. And SETs in Ads typically sit in the middle of very complex project integrations.
And we’re asked to dive in and figure out the subtle issues. And all of that is really rewarding. If you think about Ads as a whole, we serve on the order of millions of queries per second, some of the busiest systems in the world.
And it’s critical for our publishers and for users to serve correct Ads so that you can keep getting great content on the internet for free. That’s the thing, right? We look at the bigger vision, the bigger picture of what Google is about.
And we’re at the heart of that. And we’re making sure, every day we come into work, that we are moving Google’s mission forward. And we’re helping the world’s best developers develop really high-quality software.
And we deal with incredibly complex systems. And the interesting thing about the development process at Google is that you may have worked in places where you depend on version 2.0.5 of a particular software package.
And they release a new software package that you depend on. And you decide, OK, I’m going to switch to that. But that’s not really the way it works at Google. Everything is built and depends on head.
And we build it head. And we depend on head. And it’s a little scary, actually. But the systems and the automation and the tooling that we’ve put in place to help engineers do this is awesome.
And a little bit more about this. We build cross-team tools, things that detect flakiness, things that run all of our unit tests on every single change that goes into our one code base. We do a lot of really neat dashboard type stuff, machine learning type stuff.
As Karin mentioned earlier, we’ve got people working on 20% projects. Some people even work on features just to get a better feel for the system. And so all of this stuff, like tying in all this stuff together, the scale at Google at which we operate, the mission that Google is trying to fulfill, and coming in daily and interacting with some of the brightest minds in computer science, in the industry, in tech, all of that is awesome stuff.
That really can’t happen anywhere else. It’s just amazing. DIEGO: Yeah. Yeah, I’ve seen that. It’s such a great opportunity to create impact on the entire world. DAVID CHEN: Absolutely.
DIEGO: Yeah. And now to bring it more back to the day-to-day activities. Chaitali, can you tell us a little bit about what a typical day looks like for an SET? CHAITALI NARLA: Oh, that’s tough. I don’t have a typical day, usually.
Every day is different. But I’ll try to quantify things. So I think spend about 50% of my time on development work. And that includes things like design docs, code reviews, writing the actual codes for implementing things.
And then the other 50% is usually collaboration. And that could be a variety of collaboration from actually getting feedback on my design docs to incorporating the features requests that people are bringing to me and so on.
It could also be collaboration like meetings or just white boarding sessions, which is where most of our ideas start off, so that kind of information gathering and input. And this where sitting physically with the team and being part of the team, like Jonathan mentioned, really helps out.
So I would say that is what my typical day looks like. But like I said, there’s no typical day at Google. DIEGO: That’s true. Now for the people who are watching us right now and they’re wondering, hey, should I apply for an SET, do I have the skills that it takes to be an SET, why don’t you guys tell us a little bit about what are the skills, the qualities of an SET? Dave? DAVID CHEN: Sure.
I’d love to get us started here. You know, just to jump back earlier to a comment that Karin made, which I thought was awesome. SETs are really software engineers that focus on product quality, testing, automation, things like that.
And in fact, we have the exact same skill requirements in terms of coding, algorithms, data structures, that a regular SWE has. In fact, it’s exactly the same level. And if you look at the leveling across, it’s all equivalent.
But there’s an added dimension to SET that I think is a great fit for a lot of people. And they don’t necessarily realize it. So a few things, right? If you’ve coded before, you know that the tools make a huge difference.
If you’ve got good tools, you’re able to be productive. If you’ve got bad tools, you’re fighting the tools, instead of working on really cool software, right? So if tools, subtle bugs drive you crazy, think about SET as a potential fit.
If you like looking at things from a higher level perspective, a system level prospective, a data perspective, and you like thinking about, OK, data starts here, is processed– and this is petabytes or more of data, cloud computing, all of that really cool stuff that Google is known for.
If you like thinking about data, if you like thinking about obscure edge cases, systems under load, how to perform performance systems. And also how somebody externally might want to break things, or how we break things internally as we release, as we strive to do crazy, crazy things like, hey, I want to push my software to production every week.
That’s a tough challenge. Think about that, where the traditional software model is, oh, we’ll release something every six months or every year. So if those sorts of problems interest you, SET is a fantastic role to dive in and solve those problems and think about, at a meta level, how do you improve software development.
How do you really change the game and make developing software a joy, instead of having to deal with tools and having to finagle pieces that fit together. And lastly, I think a really important key characteristic of really good SETs is the ability and the desire to communicate across teams, across tech leads, across engineers, across a whole bunch of different groups, to bring people together.
And I think that’s a key characteristic as well, in addition to the strong, strong technical people we’re looking for, we look for strong people skills, when it comes to being an SET. Because you’ve got to be able to bring people together, influence people in pushing the right solution forward.
And it’s really exciting. DIEGO: Yeah. That’s a very, very good description of what an SET should have, right? CHAITALI NARLA: +1 to that. DIEGO: Now Karin, can you add a little bit about that? JONATHAN VELASQUEZ: Well, I think Dave touched on a lot of the points.
But really, he’s right. If you have a big passion for high-quality code, for helping others write the code, for solving these large problems. Again, as I also talked about earlier, it’s just as complex problem as actually developing software.
In some cases, I even think it’s more complex, because how do we test this software that has all this data? And we can’t really test on the real live data and also some things. So having a passion for quality and actually, whether it’s tools, whether it’s frameworks, but wanting to help make products better.
If you’ve ever been really annoyed with why does something work this way, why can’t they just test this and make sure it doesn’t break, SET is also a great role. And you don’t really have to have a lot of experience with test frameworks.
I think sometimes we refer to it as the adapt testability. Or if you just have the passion for these kind of things, even if you haven’t written tests or test framework in the past, that’s fine.
The only thing I had written when I started Google was unit tests. We didn’t have a lot of automation tests in my previous job. It was a lot of manual testing. So I just want to emphasize, you don’t have to have a lot of experience in writing actually automation.
But if you are interested in looking at a product, looking at the code, how can it break, work closely with both SWEs and testers in some cases to figure out how can I make everybody’s life better.
Not just the SWEs, but also the [INAUDIBLE] testers and make sure that they don’t have to do the same repetitive task every day, so we can, again, release often, weekly, daily, really a lot of impact.
So it’s a great product. Also, a great role if you want a lot of impact as well. And you can work on anything. We have people from G+, Ads, Mobile. There’s a ton of different areas, if you have particular interests.
DIEGO: Well, we have people from all those areas right here right now. KARIN LUNDBERG: And that’s just the five of us, right? DIEGO: Yeah. Exactly. All right. All right, guys. Thanks, for joining us today.
We hope you’ll learned more about what a Software Engineer in Tests role at Google is. If this is a role you guys are interested in, you can search for Software Engineer in Test on the google.com/jobs website.
Or you’re a student right now looking for an internship or a full-time job, you can go google.com/students/swenewgrad, S-W-E new grad, and indicate you’re interested in the Software Engineer in Test role.
That’s how you do it. Thank you, very much, for joining us. I’ll see you next time.