I'm in over my head at my job. What should I do? [closed]

Question

About 8 months ago I was hired as the sole developer at a small company to take on a certain project. Although there are other small projects that come up from time to time, I've been spending about 95% of my time over the last 7 months working on this project. At the rate I'm going, I think it will be at least a year and a half before I'm done with what we all hoped would be accomplished within two or three months. My boss seems to be aware of this; it would be impossible not for him to be based on what he's seen so far.

One reason the project is taking so long is because I keep having issues pop up. This project deals with millions of records stored in dozens of different tables. Although I'm as competent as anyone when it comes to databases, I'm by no means a database expert or DBA, so I can't look and a slow query and immediately say, "Oh, yes, it needs an index on X, Y and Z and we need to shard this table and we need to tweak the max_foo setting in the config." It takes me time to figure out what's going on. So far I've solved almost every significant performance issue I've come across, but there are a lot of unique challenges and I can't just become an expert overnight. And the performance issues that haven't been "worth" solving---because it would take me three days to make something run just a touch faster---are peppering my day with all these annoying little waits, where I get held up for 30 seconds but it's not like I can go do 30 seconds worth of meaningful coding somewhere else during that time.

It's possible that this is the kind of project that really deserves a team of 3 developers and a DBA. It could also be that a better or more experienced programmer could handle this project on his own in a reasonable amount of time but my current skills just aren't equal to the task. In any case I feel like I'm in way over my head. I know I can eventually get the project done but it might take a ridiculous amount of time.

It's also quite lonely and discouraging to work on a giant project by yourself with no one else to discuss it with. I'm sure at least some other developers know what I mean. There's no reason for anyone else at the company to care one tiny bit about the ins and outs of the system, and there's no reason for any developer outside of the company to care about the project. The isolation combined with the slow pace is a real motivation killer.

Any advice?

Answer 1

Every programmer I know has been in your spot at least once. I know I have. It's a killer, for sure. The way I get to feel better is to take the problem to the boss, explaining it to him in exactly the vivid words you've so forcefully and heartbreakingly used here in your post.

Yes, of course your boss has noticed but he might not know that it's not your fault. He wants to know -- he needs to know -- what's going on with you. And he has a right to know, in my opinion.

If you can tell him, openly and honestly, what the technical stumbling blocks have been I believe that he will respond in a helpful way. Maybe you can get better tools, or some part-time consulting help on the DBA part, or even some time off so you can re-introduce some of your old interests into your software-burdened life.

No real boss wants a discouraged and -- face it! -- partly paralyzed contributor. It's a bitch, I know, and it's hard to go to the boss with this story (particularly, you might think, this late in the project -- why the hell couldn't I tell him sooner, damn it!) but it will be damaging to you if you don't. At the least, you won't feel quite so discouraged.

If he is not encouraging, well, at least you can cut your losses and get out of there.

Answer 2

I'm tempted to say "learn how to do performance tuning" for which there is plenty of good advice here and on stackoverflow.

But there's a bigger issue here. You need to lay it all out to your boss, and give him/her options, such as bringing in a consulting DBA. Never hide anything from your boss - it will only reflect badly on you when the stuff hits the fan. If you're in over your head, it's probably not your fault. In any case, that's what managers are for.

Answer 3

I was in a similar situation early in my career. Solo developer, etc. What we ended up doing is bringing in a hard core database expert for a month to get the database tuned and restructured. It will cost a bit to do that, but in the long run it saved us a ton of time which saved us money.

As to not having people to talk to or bounce ideas off of, you'll probably have to do that on your own time. Forums, user groups, SE, etc.

Answer 4

My advice is that you stick with it. If your company is aware of the problems you are facing and the amount of work resting on your shoulders and they are okay with it, then I don't see the issue. You could suggest for them to hire another developer or DBA if you really must get things moving quicker, but if they are okay with the amount of time you will need to complete the project, then just keep working at it. You will learn a lot, trust me.

Answer 5

Learn how to read an execution plan (SHOWPLAN / EXPLAIN / etc.).

No, seriously; I don't mean to offend, but it should not take you 3 days to optimize a query or index.

If it does, then I suggest you (a) use the vast resources available to you on the internet to teach yourself how, or (b) ask your employer to send you out on a training course. If a 1-week course is going to triple your productivity, then they should be happy to pay the $5000 or whatever it costs.

If the performance issues are design- or architecture-related then that's a different issue. Obviously you need to communicate that to your employer/manager. The sooner the better. Don't try to be a hero. If the system you're working on is demonstrably a piece of junk then - assuming you have some proof of this - you should have no problem managing expectations and/or getting extra help. Just communicate.

Answer 6

I have been in your situation before, in a team of two coding on a large project that somehow grew outside our handling. We also had an large database, lots of requests which resulted in quite a stressful project.

We ended up hiring a database expert for a while, working besides us at the project. Not only was this cheaper in the long run, it educated us alot creating even more value for the company.

Answer 7

_{Different from everybody else, I was on the very same situation and I got fired after 1 month of failed attempts to do the job of a DBA. Granted, my situation was peculiar for few other reasons, but maybe I can add a very different perspective here.}

Repeating everyone else: talk to your boss. And talk to your co-workers. Give daily reports. Let it clear your situation to everyone. Show the effort. Most of the times, that's all that really matters.

_{In my case I didn't do it enough. But it wouldn't be enough anyway - I was just not getting paid well enough for all the pressure and requirements needed. I failed at accounting that along with too many things in my life happening all at same time.}

Thus, another thing to consider is how much you're getting paid and how good your personal life is in general. It will be much harder to take the pressure if you're not very well structured - get your own home, get someone back at home just to be there when you get a cold or resistances down for any reason, stuff that many people take for granted. If you're in the kind of project that needs extra hours and weekends, you'll need every support you can get - and that will not be accounted in yours or anyone's salary.

Answer 8

I have no idea what language you're using etc. but have you considered pointing to a different, vastly smaller database (in terms of # of rows of data)? It's not uncommon to have a separate db instance for dev, qa, and production. If you can do this, you take database performance issues (and presumably bad data issues) out of the picture and you can focus on developing the core functionality of your app, whatever that is.

It's entirely possible that your boss can hire a separate DBA to optimize the indexes and queries, and that can be done in parallel to your work.

EDIT:

To answer this in a more big picture way, you should schedule a meeting with your boss. Tell him that you'd like to do things slightly differently. Break your project down into tasks and provide estimates for each task (try to make the tasks completable in about 1-2 weeks). If you can't estimate a task, tell your boss. Tell him you're providing these estimates so you both can come up with a realistic project timeline. Tell him why you can't estimate certain tasks ("I don't have the background in this area so I can't provide a realistic estimate."). Ask him for help in coming up with solutions to these blockers (hire a DBA, send you to training, etc.). Make sure each task is a deliverable that can be demoed to your boss. Schedule a time every 1-2 weeks to actually demo the feature. If you miss an internal deadline on a task explain to your boss why it was missed, how to improve process, and what your new estimate is.

If you do these things your boss will have no reason to be unhappy with your progress. Even if it takes a year, it's a timeline he agreed on, and you'll be delivering on schedule every 1 to 2 weeks. He'll see the incremental results each week.

There's a very real chance that the timeline given (2-3 months) was an estimate that was arrived at without much thought and with some vague hand waving. So essentially you're re-negotiating the timeline and getting buy-in from your boss.

Answer 9

Easier said than done, but stop worrying about schedule problems. That's your manager's job, not yours. Your job is to do as much as you can do, make sure your manager doesn't just "seem to be aware" but explicitly spell out your issues and estimates in status reports, and to tell your manager what you would need from him if he asks you to do it faster.

If he doesn't give you what you need, you can assume he has considered the business implications and is willing to trade off the deadline for the lower short term costs. This is frequently the case in small business/lone developer situations. Good managers consider schedule slips a technical hurdle rather than a sign of incompetence, and software development is notoriously difficult to estimate.

Answer 10

I hear you. I was in a similar situation just over a year ago.

You must tell your boss exactly how you feel, just as you have explained it to us. Plan out what you are going to say, and ask him for a chat when he has some time free (tell him how long you expect it'll take so he can set aside the time for it).

As others have said, tell him exactly what you are finding difficult. If he's technically minded, tell him in detail exactly what is difficult. If not, don't throw jargon at him but try to make him understand what aspects of the job are difficult.

Provide him with some possible solutions. You've already recommended he hires a DBA. Is that enough? Have you made it clear that the company (and you!) have an urgent need for extra staff - maybe another developer as well, not just a DBA? Unless you make it clear it's got to breaking point, he may not realise the urgency it so obviously has.

You sound like you're feeling the way I did a while back. It -really- got to me, and actually affected my life outside work. My relationship with my wife suffered and once I realised that, I just had to change jobs. I tried to leave work at work, but it's so much easier said than done when you know you're going to be back in that hellish situation in just a few more hours. My advice: take action now. Don't wait any longer. Talk to your boss, give him a chance to do something about it. If nothing can be done, or won't be done, then you really must look out for yourself and change jobs. I was extremely lucky and got offered a job with a far bigger company with more experience and support for the employees (I went from a local company of ~10 people, only 3 of which were in the office I was in to an international company of tens of thousands).

Keep your sanity. Don't let it get to you. You will get through this, one way or another.

Good luck.

Answer 11

See if your boss knows a DBA in another company who owes him a favor, or would grant him a favor, have that DBA sit with you or over conference call for a couple 1-2 hour sessions. I was in over my head(I'm a developer tasked with building out a drupal infrastructure--database crashes, and another task to start up replication) and just a couple sessions with a real DBA helped a lot. Also solves your isolation problem. After just those 2 sessions I went all command line and wouldn't touch phpmyadmin anymore, no need.

Answer 12

That is part of life in IT. And that is why 99% of the drones don't volunteer to do anything new.

This does not make you a bad person or a loser. It does show you and everyone in the company that you cannot do this job.

Your escape hatch has passed. You should have never got into this situation or jumped out as soon as you saw the technology was over your head.

I have seen people in your shoes - and they all got fired.

Find another job while you still have this one - that is your only option. And learn from your mistake of taking on something that you cannot do.

Answer 13

When you wake up everyday, just say "This is a brand new day, and I will make the best of this day". Before coming to the office, which has crappy coffee, head to Starbucks or your favorite coffee place, get an Americano and start coding. It will help you succeed.

Embrace the challenge you get at work and make the best of it.