Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

We are currently using Entity Framework as an ORM across a few web applications, and until now, it has suited us well as all our data is stored in a single database. We are using the repository pattern, and have services (the domain layer) which use these, and return the EF entities directly to the ASP.NET MVC controllers.

However, a requirement has come up to utilise a 3rd party API (through a web service) which will give us extra information that relates to the user in our database. In our local User database, we will store an external ID which we can provide the to API to get additional information. There is quite a bit of information available, but for the sake of simplicity, one of them relates to the user's company (name, manager, room, job title, location etc). This information will be used in various places throughout our web apps - as opposed to being used in a single place.

So my question is, where is the best place to populate and access this information? As it is used in various places, it's not really sensible to fetch it on an ad-hoc basis wherever we use in the web application - so it makes sense to return this additional data from the domain layer.

My initial thought was just to create a wrapper model class which would contain the EF entity (EFUser), and a new 'ApiUser' class containing the new information - and when we get a user, we get the EFUser, and then get the additional info from the API, and populate the ApiUser object. However, whilst this would be fine for getting single users, it falls over when getting multiple users. We can't hit the API when getting a list of users.

My second thought was just to add a singleton method to the EFUser entity which returns the ApiUser, and just populate it when needed. This solves the above problem as we only access it when we need it.

Or the final thought was to keep a local copy of the data in our database, and synchronise it with the API when the user logs in. This is minimal work as it's just a synchronisation process - and we don't have the overhead of hitting the DB and API every time we want to get user information. However, these means storing the data in two places, and also means the data is out of date for any user that hasn't logged in for a while.

Does anyone have any advice or suggestions on how best to handle this kind of scenario?

share|improve this question
    
it's not really sensible to fetch it on an ad-hoc basis -- Why? For performance reasons? –  Robert Harvey Dec 12 '13 at 18:58
    
I don't mean hitting the API on an ad-hoc basis - I just mean keeping the existing entity structure as it is, and then calling the API ad-hoc in the web application when needed - I just meant this wouldn't be sensible as it would need to be done in lots of places. –  stevehayter Dec 12 '13 at 22:29

3 Answers 3

up vote 2 down vote accepted
+50

Your case

In your case all three options are viable. I think that the best option is probably to sync your data sources someplace the asp.net application is not even aware of. That is, avoid the two fetches in the foreground every time, sync the API with the db silently). So if that's a viable option in your case - I say do it.

A solution where you make the fetch 'once' like the other answer suggests doesn't seem very viable since it doesn't persist the response anywhere and ASP.NET MVC will just make the fetch for every request over and over.

I'd avoid the singleton, I don't think it's a good idea at all for plenty of the usual reasons.

If the third option is not viable - one option is to lazy load it . That is, have a class extend the entity, and have it hit the API on a need to basis. That's a very dangerous abstraction though since it's even more magic and non-obvious state.

I guess it really boils down to several questions:

  • How often does the API call data change? Not often? Third option. Often? Suddenly the third option is not too viable. I'm not sure I'm as against ad-hoc calls as you.
  • How expensive is an API call? Do you pay per call? Are they fast? Free? If they're fast, making a call each time might work, if they're slow you need to have some sort of prediction in place and make the calls. If they cost money - that's a big incentive for caching.
  • How fast does the response time have to be? Obviously faster is better, but sacrificing speed for simplicity might be worth it in some cases, especially if it's not directly facing a user.
  • How different is the API data from your data? Are they two conceptually different things? If so, it might be even better to just expose the API outside rather than return the API result with the result directly and let the other side make the second call and handle managing it.

A word or two about separation of concerns

Allow me to argue against what Bobson is saying about separation of concerns here. At the end of the day - putting that logic in the entities like that violates separation of concerns just as bad.

Having such a repository violates separation of concerns just as bad by putting presentation centric logic in the business logic layer. Your repository is now suddenly aware of the presentation related things like how you display the user in your asp.net mvc controllers.

In this related question I've asked about accessing entities directly from a controller. Allow me to quote one of the answers there:

"Welcome to BigPizza, the custom Pizza shop, may I take your order?" "Well, I'd like to have a Pizza with olives, but tomato sauce on top and cheese at the bottom and bake it in the oven for 90 minutes until it's black and hard like a flat rock of granite." "OK, Sir, custom Pizzas are our profession, we'll make it."

The cashier goes to the kitchen. "There is a psycho at the counter, he wants to have a Pizza with... it's a rock of granite with ... wait ... we need to have a name first", he tells the cook.

"No!", the cook screams, "not again! You know we tried that already." He takes a stack of paper with 400 pages, "here we have rock of granite from 2005, but... it didn't have olives, but paprica instead... or here is top tomato ... but the customer wanted it baked only half a minute." "Maybe we should call it TopTomatoGraniteRockSpecial?" "But it doesn't take the cheese at the bottom into account..." The cashier: "That's what Special is supposed to express." "But having the Pizza rock formed like a pyramid would be special as well", the cook replies. "Hmmm ... it is difficult...", the desparate cashier says.

"IS MY PIZZA ALREADY IN THE OVEN?", suddenly it shouts through the kitchen door. "Let's stop this discussion, just tell me how to make this Pizza, we are not going to have such a Pizza a second time", the cook decides. "OK, it's a Pizza with olives, but tomato sauce on top and cheese at the bottom and bake it in the oven for 90 minutes until it's black and hard like a flat rock of granite."

(Read the rest of the answer, it's really nice imo).

It's naive to ignore the fact there is a database - there is a database, and no matter how hard you want to abstract that, it's not going anywhere. Your application will be aware of the data source. You won't be able to 'hot swap it'. ORMs are useful but they leak because of how complicated the problem they solve is and for plenty of performance reasons (Like Select n+1 for example).

share|improve this answer
    
Thanks for your very thorough response @Benjamin. I initially started creating a prototype using Bobson's solution above (even before he posted his answer), but you raise some important points. To answer your questions,: - The API call isn't very expensive (they're free, and also fast). - Some parts of the data will change fairly regularly (some even every couple of hours). - Speed is fairly important, but the audience of the application is such that lightening fast loading isn't an absolute requirement. –  stevehayter Dec 17 '13 at 10:28
    
@stevehayter In that case I would most likely perform the calls to the API from the client side. It's cheaper and faster, and it gives you finer grained control. –  Benjamin Gruenbaum Dec 17 '13 at 10:32
1  
Based on these answers, I'm leaning less towards keeping a local copy of the data. I'm actually leaning towards exposing the API separately, and handling the additional data that way. I think this may be a good compromise between the simplicity of @Bobson's solution, but also adds a degree of separation which I'm a little more comfortable about. I will look at this strategy in my prototype, and report back with my findings (and award the bounty!). –  stevehayter Dec 17 '13 at 10:37
    
@BenjaminGruenbaum - I'm not sure I follow your argument. How does my suggestion make the repository aware of presentation? Sure, it's aware that an API-backed field has been accessed, but that doesn't have anything to do with what the view is doing with that information. –  Bobson Dec 17 '13 at 12:47
1  
I opted to move everything to the client-side - but as an extension method on the EFUser (which exists in the presentation layer, in a separate assembly). The method simply returns the data from the API, and sets a singleton so it isn't hit repeatedly. The lifetime of the objects is so short that I don't have a problem using a singleton here. This way there is some degree of separation, but I still get the convenience of working with the EFUser entity. Thanks to all the respondents for their help. Definitely an interesting discussion :). –  stevehayter Dec 20 '13 at 11:29

With proper separation of concerns, nothing above the Entity Framework / API level should even realize where the data is coming from. Unless the API call is expensive (in terms of time or processing), accessing data that uses it should be as transparent as accessing data from the database.

The best way to implement this, then, would be to add extra properties to the EFUser object which lazy-load the API data as needed. Something like this:

partial class EFUser
{
    private APIUser _apiUser;
    private APIUser ApiUser
    {
       get { 
          if (_apiUser == null) _apiUser = API.LoadUser(this.ExternalID);
          return _apiUser;
       }
    }
    public string CompanyName { get { return ApiUser.UserCompanyName; } }
    public string JobTitle{ get { return ApiUser.UserJobTitle; } }
}

Externally, the first time either CompanyName or JobTitle is used there will be a single API call made (and thus a small delay), but all subsequent calls until the object is destroyed will be just as fast and easy as database access.

share|improve this answer
    
Thanks @Bobson... this was actually the initial route I started going down (with some extension methods added in to bulk load the details for lists of users - for example, displaying the company name for a list of users). It seems to to suit my needs well so far - but Benjamin below raises some important points, so I'm going to continue evaluating this week. –  stevehayter Dec 17 '13 at 10:16

One idea is to modify ApiUser to not always have the extra info. Instead, you put a method on ApiUser to fetch it:

ApiUser apiUser = backend.load($id);
//Locally available data directly on the ApiUser like so:
String name = apiUser.getName();
//Expensive extra data available after extra call:
UserExtraData extraData = apiUser.fetchExtraData();
String managerName = extraData.getManagerName();

You could also modify this slightly to use lazy loading of extra data, so that you don't have to extract UserExtraData from the ApiUser object:

//Extra data fetched on first get:
String managerName = apiUser.lazyGetExtraData().getManagerName();

This way, when you have a list, the extra data won't be fetched by default. But you can still access it while traversing the list!

share|improve this answer
    
Not really sure what you mean here though - in backend.load(), we're already doing a load - so surely that would load the API data? –  stevehayter Dec 20 '13 at 11:24
    
I mean that you should wait doing the extra load until explicitly requested - lazy load the api data. –  Alexander Torstling Dec 25 '13 at 11:34

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.