Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

A company I'm going to interview with says they deal with huge amounts of data. How does one prepare for an interview where that fact is telegraphed beforehand as something that one really ought to ponder over?

Should I memorize maximum file sizes for Ext4 file systems or something? Or just be aware of the need for the best possible algorithms...

Thanks.

share|improve this question
2  
Doesn't it depend on what they're doing with that data? –  user16764 Jul 7 '11 at 22:56
1  
They move it from point A to point B. They do streaming. –  Ponk Jul 7 '11 at 22:59
1  
@CodeMonkey You should edit your question to include that additional bit of information. –  Anna Lear Jul 7 '11 at 23:02
1  
What job are you being interviewed for? What will be your role? What is the company about? ... very vague question. The only concrete piece of information is "huge amounts of data" and even that ("huge") is relative nowadays. –  Rook Jul 7 '11 at 23:09
add comment

5 Answers

I worked at a storage company and it sucked immensely. Basically data is stored on large hard drives which are attached to servers. In addition they attempted to have a clever scheme for data integrity/security/recovery etc. Hard drives are dirt-cheap, but they are very likely to die within 5 years. So, how do you keep the data safe?

Google came up with this back in 2003:

http://en.wikipedia.org/wiki/Google_File_System http://labs.google.com/papers/gfs.html

The company that I worked for never got even close. As the result hospitals lost important documents, CIA lost track of terrorists, banks lost customer data - I exaggerate not!

I highly recommend reading the Google File System paper. They did a good job at explaining stuff.

Also, be able to compare different file systems (at a high level): ext3, ext4, reiserFS, NTFS, the one that Apple uses ...

Also try to learn what a company Akamai does and why they are in business - who would pay them and why.

Knowing something about networking, caching, proxy servers, BGP tables http://en.wikipedia.org/wiki/Border_Gateway_Protocol would help.

Look up what the company does, try to figure out who their customers are, what problem they have and why they pay $$$ to your employer. Then try to imagine what low-level details are involved in streaming stuff from point A to point B, including protocols, etc.

Hopefully this will help you 'WOW' them.

EDIT: Do not be cocky at the interview, even if you happen to know some things that they do not, do not show off. You are trying to get a job, right?

I realize that my answer is not complete, albeit somewhat popular.

I would add knowing the difference between TC and UDP, and when you would need each one.

share|improve this answer
1  
+1 - research is your friend –  Justin Shield Jul 8 '11 at 12:08
    
+1: Know your domain. –  Richard DesLonde Jul 8 '11 at 20:46
add comment

Safe bets would be network topologies and technologies, as well as application architectures. High-volume filtering and routing techniques might also be good. If their primary business involves serving or storing data from external customers, then security should be involved. Showing some database knowledge is probably a good idea too. If their data is transactional, then you should know basic transaction semantics.

share|improve this answer
add comment

It depends on the job.

If you're doing database work, you should know your joins so you don't create a massive Cartesian Join / bog down the system. This is a big problem with some of the developers at my workplace.

Knowing how to string together subqueries and writing efficient code is always a plus. Our team prefers programmers who don't need to test every single snippet on the db2 server. Being confident in your code prior to testing keeps you in good graces on my team.

share|improve this answer
add comment

I would take this with a pinch of salt unless you get some concrete figures such as number of Terabytes and number of locations.

In may experience most sites think they have a "huge" amount of data because they are inward looking and have no idea what the rest of the world is doing. Currently I am looking into a system with a billion rows in the database, and I would classify this as a "medium" to "large" system.I have worked on systems where the transaction history alone held millions of rows.

Never trust terms like "huge", "large", "high volume" etc. always ask for concrete numbers. In most cases these will turn out to be rather modest.

share|improve this answer
add comment

We at our company deal with a lot of data (social stuffs), what I have learned is:

  • know how hardware deals with data and how does the OS (kernel), know the bottlenecks/limits.
  • have a good understanding over the data you are dealing with, this helps you to understand and solve problems faster and easier.
  • learn at least basic algorithms and read best practices for dealing with data.
  • don't be afraid it's just data :)

I know this is not the answer you have expected but I hope it would be of some help and good luck.

share|improve this answer
    
Luckily these are topics that interest me. I don't know whether I have a shot at this job (I'm a nervous intervieweee) but I'm very much into learning about bandwidth issues, cache utilization etc. Thanks. –  Ponk Jul 7 '11 at 23:53
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.