Codassium logo

Thoughts on coding, software development & hiring.

My framework for generating great interview problems

If you interview candidates regularly, you would like to have a good pool of questions just to break the sheer monotony of asking the same question over and over again. There is no shortage of posts on the Internet about what coding challenges and questions to ask candidates during various stages of the technical screening process. Instead, I want to share the framework that I use for coming up with interview questions that allow me to capture and evaluate a candidate’s thought process.

Here’s what I do

Every time I solve an interesting problem at work, I try to document my experience and make an attempt to see how I can model the challenge I faced as an interview question.

Let’s take a specific example. Not long ago, I had to implement measures to prevent a location API (which returned metadata about a given latlong) from being used by unscrupulous companies for scraping and re-selling of data. The solution involves a number of architectural as well as algorithmic components, so it allows me to explore the breadth and depth of the solution, depending on the candidate’s experience and areas of strength.

Why this framework works for me

I believe that good interview problems should be centered around issues which you yourself have encountered and solved previously.

  • You would understand the problem space deeply. This is critical for evaluating the answers, and to ask follow-up questions on peripheral areas.

  • You would have gone through the complete life cycle of the solution - right from exploring potential approaches, to evaluating the pros and cons of each solution and finally narrowing down and solving the problem. This helps you compare how much the candidate’s thought process is aligned to your own thought process, as you went about solving the problem.

  • Imaginary puzzles and problems are just that - imaginary. They don’t really test the candidate’s ability to perform in your environment. Picking a problem that you have solved recently would help you get a glimpse of whether the person would be a fit for your organization, solving the kind of problems you solve at work everyday.

  • Another nice benefit of this method is that you are now simulating a typical work environment where the candidate and you are working together to solve a problem and you can get a sense of how opinionated the candidate is: i.e., if they are flexible and open-minded to consider others’ inputs (the interviewer’s nudging) or if they ignore all external inputs and just chug along with what they think is the best solution. This can surface potential team fit issues.

  • At the end of the interview, I’ll also tell the candidate that this is one example of the type of problems we solve in the company. This gives the candidate a good overview of the type of work that’s done in the company (so your example better be interesting and challenging!).

There is a gotcha, though

There is only one thing to be cognizant of, though: don’t expect the candidate to arrive at your eventual solution right away, without having the benefit of going through the exploration and evaluation phases that you yourself would have gone through. Remember that this question, being fairly open-ended, is not about giving the perfect answer - it should be a dialog that captures the day-to-day problem solving you would expect somebody to do as a potential colleague.

The forgotten comma operator

Most of us are familiar with comma as separators in C/C++.


    int add(int a, int b)                         // parameter list
    for(int i=0, j=list.size(); i < j; i++)      // variable declaration list
    MAX(2, 3)                                    // macro argument list

What’s lesser known is that the comma is also an operator that has the lowest level of precedence amongst all operators. The comma operator evaluates its operands from left to right and returns the value of the last operand.

Let’s take an example.


    int result = (2, 3);   // result is 3

The evaluation proceeds this way:


    (2, 3) evaluates to 3
    result is then assigned 3

When we wrap the RHS with parenthesis, we give it the highest order of precedence. In fact, if we omitted the parenthesis, the code will simply fail to compile.


    int result = 2, 3; // error: expected unqualified-id before numeric constant

Practical uses

The comma operator allows you to specify more than one expression where the language expects only one. Unfortunately, since the usage of comma as an operator is so poorly understood, I have seen succinct code being labelled as confusing because of the comma.

The comma operator is handy when you want to increment more than a single variable in a loop. Let’s say you want to split a list of numbers into roughly equal sized chunks, you can write something like this:


  const size_t LIST_SIZE = numbers.size();
  const size_t CHUNK_SIZE = (LIST_SIZE / NUMBER_OF_CHUNKS) + 1;

  // `i` and `chunkIndex` are both incremented in the outer loop
  for (int i = 0, chunkIndex = 0; i < numbers.size(); i += CHUNK_SIZE, chunkIndex++) {
    for(int k = 0; k < CHUNK_SIZE && i+k < LIST_SIZE; k++) {
      // type of `chunks` is vector<vector<>>
      chunks[chunkIndex].push_back(numbers[i+k]);
    }
  }

Notice how i and chunkIndex are both incremented in the outer loop?

You can also use the comma to pack multiple statements into a ternary operator. Usually, if else is the right way to go if you want to conditionally execute multiple statements, but sometimes the ternary operator along with the comma provides a succinct alternative. Here’s a synthetic example:


    (i > 0) ? (i--, doSomething()) : doSomethingElse();

Overloading Comma

C++ allows you to overload the comma operator. Boost Spirit uses this to a great effect for implementing list initializers for symbol tables.


    symbols<> logicalOperators = "AND", "OR", "NOT";

Spirit achieves this by overloading the equal and comma operators. So, the above statement is identical to:


    logicalOperators.operator =("AND").operator ,("OR").operator ,("NOT");

The first operator =() method returns an object on which the other operator ,()s are invoked. Interestingly, remember how I mentioned that the the parenthesis is mandatory in the very first example? It turns out that in this particular case, wrapping the list of operators with a parenthesis will not work!


    symbols<> logicalOperators = ("AND", "OR", "NOT");  // will fail

With a parenthesis, the compiler just sees the comma operator between two char*, which cannot be overloaded.

Comma in JavaScript

JavaScript has inherited the comma operator from C, and is often used by minifiers. For example,


    if(x) {
      foo();
      return bar();
    } else {
      return 1;
    }

will be minfied to


    return x?(foo(),bar()):1;

In ECMAScript 6, the comma is also used for destructuring assignment. You can use Firefox’s web console to tinker with this.


    var [x, y] = [1, 2];
    x === 1; // true
    y === 2; // true

And, that brings us to the end of a short essay on the forgotten operator!

Analyzing YC14 companies' hosting decisions

A few years ago, Joel had published a chart summarizing the hosting decisions of YC companies.

I wanted to find out how things have changed since then, so here’s the breakdown of the hosting decisions of 107 YC14 companies taken from YCUniverse.

Domain Registrars

Given the negative sentiment about GoDaddy that’s so prevalent on HN, I was surprised to find GoDaddy still being preferred by one-thirds of the YC14 start-ups. My original prediction here was about eNom (and its resellers like Namecheap) dominating, but that’s not to be.

Domain registrars

Hosting

My original prediction here was AWS being used by 75% of the start-ups, but the actual number turned out to be a touch lower. More surprisingly, a significant number of YC14 companies seem to be hosted on GoDaddy.

Hosting

SSL Certificates

About 10% of the YC companies do not have a HTTPS enabled website (not that this by itself is to be always frowned upon). Among the ones that did, some of them did not redirect non-HTTP requests to HTTPS.

SSL Certs

You can download the raw data here.

Good code seldom emerges right away

I often see people struggle to write good code from line number one. Instead of making progress on a problem they need to solve, they get caught up in designing the solution the “right way”, tripping over a myriad classes, interfaces and what not. This is especially the case with those who are new to programming - they have been taught about the importance of clean code, and try their best to follow what they have learned.

While the intention here is right, good code seldom emerges right way. Good design and good code, evolves from bad code, just like how a diamond emerges from the rough. Trying to write perfect code from the start often leads to over-engineering.

It’s not easy to build both the desired functionality and the correct design for it simultaneously. In many cases, the desired functionality itself evolves as you start implementing a solution. We don’t have to place this artificial constraint on us at all.

It’s easier to have two modes of operation when you’re coding. In the first mode, you will just bang out some code that nails the basics of the desired behavior. Once you’re fairly certain about the direction you’re going to take, write some tests around your functionality and switch to the software-engineering mode to refactor and redesign the internals. The tests will help to ensure that your refactoring does not break the correctness.

This approach is not an excuse to write bad code. It merely separates the problem solving and the design concerns so that one doesn’t need to focus on both of these at the same time. It’s important to emphasize the importance of refactoring as an important step towards the final solution and ensure that bad code is not shipped to production and conveniently forgotten.

Stop asking about stuff one doesn't need to get the job done

I used to work at a company that primarily developed web and mobile applications. To excel in that place, one needed familiarity with a few common frameworks, a good understanding of HTTP and REST and an ability to write code that’s not a ball of spaghetti. Daily work involved gluing together various web services with some diligent unit and integration tests. Nothing fancy pancy. And yet, if you were to interview at that company, you will be asked to explain in detail red-black trees, solutions to programming puzzles and space-time complexities of obscure algorithms.

The use of brainteasers in technical interviews has come under a lot of attack lately, but that’s just a symptom of a larger issue that plagues many technical interviews: testing for skills that aren’t critical to the role being interviewed for. Blame the much advertised hiring processes in companies like Google and Facebook. I see a lot of smaller companies modeling their interviews in the same mould, without really thinking about what will work for them. The end result? The seemingly rock solid new hires struggling to get their day-to-day stuff done.

Time for some blasphemy: one does not always need to know the nuts and bolts of hard-core computer science concepts to be a productive web or iOS developer. While having a strong grasp of CS fundamentals is no doubt desirable, it should not be the focus of technical interviews as I so often hear from friends. Instead of rejecting someone right away for not being able to explain an obscure data structure, or to derive the time complexity of a given algorithm, you should instead find out whether the candidate thinks that knowing such things is a drag and a waste of time. The latter is definitely a cause for concern and is a much more valid reason for turning down somebody.

Why We're Building a Better Way to Screen Developers

Far too often we’ve seen candidates who look good on paper not live up to their claims during an actual interview. Phone screens solve this problem by giving you a chance to talk to the candidate before investing any more time or money in the process. However, a simple phone screen can be misleading. A good developer can be easily overlooked if she is not good at communicating over a phone and a bad candidate can quite easily give an impression of being good. So a simple phone screen essentially tests more of phone communication skills than a developer’s primary role - coding. It also strips out an important part of the human communication process - facial expressions or more generally body language.

A phone screen combined with a shared coding session can do wonders to remedy the first problem of testing the wrong skill-set. Companies like Facebook and Google already do this. However, this still does not solve the other problem of missing out on visual cues during an interview - does the candidate look clueless, is she receptive to hints, is the interviewer looking disapprovingly at the candidate’s current approach, etc. A video call combined with a shared coding session would remedy this problem as well as the first. If cost wasn’t a factor, given a choice between interviewing a candidate over phone and interviewing the candidate in-person, we would definitely prefer interviewing in-person since it literally gives better visibility to both the candidate and the interviewer and provides a better opportunity to engage in a holistic conversation. Of course, since cost is indeed a factor that cannot be ruled out for several businesses, the next best thing is a video conference.

To set up a video conference with a candidate, you’d probably exchange Skype IDs with the candidate, and either one of you adds the other as a contact and then initiates the call. You’d then share a link over IM to probably a Google doc or something like collabedit or the several other real-time editors out there. Meanwhile, Skype’s having a particularly bad day and the call drops and you reach for the Skype window to restart the call… It can be quite a mess to manage all of this.

Enter Codassium - Real-time code editor meets video conferencing.

Codassium is an integrated solution that offers a real-time code editor along with code execution and one-click in-browser video conferencing. You can even invite several of your colleagues to join the interview / video conferencing session. It also offers a platform to schedule interviews, send out calendar invites and track interviews. Codassium is a one-stop solution for your video-code screens. Say good-bye to archaic phone screens.

Take it for a spin - signup for the beta here: http://codassium.com