Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I work for a company that has a hosted .net internet application with many clients. Those clients often want to write customizations for our application. We have APIs to hook into the app, but the customizations themselves are written in .net. This is a shared, secure hosting environment and we have to code review these customizations before we can deploy them in our datacenter to ensure that they don't degrade performance, crash our servers, or open any security vulnerabilities. We charge for these code reviews.

The current pricing model is simply a function of the number of lines of code. I think this is a bad idea for a variety of reasons, but primarily because, if we are interested in verifying that the code works as expected, we should be incentivizing good, readable code, not compaction.

I would like to propose a pricing model that incorporates some, or all of the following as inputs:

  • Lines of code
  • Cyclomatic complexity
  • Avg function length
  • # of functions

Are there any other metrics I should incorporate, or other ideas for how we can reasonably create pricing for code reviews that encourages safe and understandable code?

share|improve this question

closed as too broad by Snowman, MichaelT, durron597, Kilian Foth, Bart van Ingen Schenau Apr 29 '15 at 13:20

There are either too many possible answers, or good answers would be too long for this format. Please add details to narrow the answer set or to isolate an issue that can be answered in a few paragraphs.If this question can be reworded to fit the rules in the help center, please edit the question.

Does each customer have their own instance of the app? – ozz Feb 2 '11 at 20:01
How does the API work - is it purely trigger driven from your end or does the client code get to start interactions (between your code and theirs) – Murph Feb 2 '11 at 20:16
Time. Bill hourly. Improving all those things areas will reduce the time you need to spend reviewing code. – CaffGeek Feb 2 '11 at 20:20
easy way to cheat people billing by LOC: perl -p -e 's/\s+$/ /g' file – TZHX Feb 3 '11 at 10:32
up vote 5 down vote accepted

Frankly, I think thats awfully brave of you to offer that option in a shared hosting environment. Are there databases in use too that are shared?

If this was a company I worked for, I'd suggest scrapping completely the notion of paying per line or other metric, and replacing it with flat rate fees based on what functionality was being used. Complex pricing structures are a 'business smell' (if I may steal a term from programming).

share|improve this answer
Without going into much detail about the environment itself, we can generally prevent any truly awful things from happening. And DBs aren't shared. The "nature" of the customization may be a better rule of thumb than anything so algorithmic. I like that idea. Combined with some basic coding guidelines could yield a pretty reasonable model. – Chris Clark Feb 2 '11 at 20:29

Test coverage would be a minimum. It should include tests that are designed to look for the problems you site security, crashes, and performance. This would be in addition to the static code evaluation criteria you have mentioned.

Your price for this level of certification should be based on the clients customization requirements. You should prepare a certification plan that outlines both the static and dynamic evaluations that will be performed.

share|improve this answer

Test coverage may be another area that would be worth adding as that can help when new things are added to ensure things aren't broken.

Design pattern use may be another idea to help with making the code be better organized at times as there are times where this can be quite useful in helping to make code that is understandable.

The tests provide an initial documentation of some of the functionality of the code which may be useful if you someone is going to be digging into the code in a sense. I'm not sure how thorough into the code you want to get with your reviews but the idea for me would be that the tests may provide a good starting point for getting into the code. While this is a somewhat optimistic view to have, sometimes life can work out that way.

share|improve this answer
Yes, the amount of regression testing for existing features should also be factored in. – FrustratedWithFormsDesigner Feb 2 '11 at 19:49
Test coverage is an interesting one. I think I'd shy away from any kind of pattern recommendations because these are disparate, 3rd party developers and I'd rather keep things metrics-driven, rather than get dragged into a religious war about whether dependency injection is really appropriate for customization xyz, or whatever. – Chris Clark Feb 2 '11 at 19:54
Also, do you think that test coverage actually lowers the overhead of performing a code review, or does it simply ensure that any future changes to the customization will be easier to verify? I'm not convinced that test coverage materially changes how easy the code is to review. Could definitely be wrong though. – Chris Clark Feb 2 '11 at 19:55

GrandmasterB has the right idea- if you start haggling with customers over LOC or avg function length (which is just LOC in a pretty dress) you are asking for trouble. Look instead to billing for the utilization of APIs/resources, security analysis and test coverage. If for instance they are using an API to do extensive updating of the database your reviews on a per function/method basis should be more rigorous/expensive than if all they are doing is extracting data for additional presentation.

An updating API utilization code review will have

  • a significant security review component
  • an in depth test coverage review
  • a thorough standard code review on a per function/method basis

By contrast if all they are doing is retrieving info for their presentation layer, your security and test coverage review can be significantly limited and less expensive. You are not after all their QA department. Your concern is that they do not implement anything that can impact your shared environment.

share|improve this answer

The hourly rate seems fair (GrandmasterB). A customer could send the code and you could provide a quote. Through experience, you should be able to determine how long it will take. Some of the metrics you mentioned could be used as an internal guideline, but you don't want customers to think they can 'game' the system. You don't pay your programmers by line of code?

Just like like the postal service, you can charge extra for getting it done faster.

Most application providers discourage customizations by charging high fees. Your company may see the ability to add your own code as a major feature of your app/service. There are more costs to your development of this application than just code review. Upgrades have to be very messy and you have no way to guarantee the customizations will still work. Hope you don't start a trend ;)

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.