Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

When designing a RESTful interface, the semantics of the request types are deemed vital to the design.

  • GET - List collection or retrieve element
  • PUT - Replace collection or element
  • POST - Create collection or element
  • DELETE - Well, erm, delete collection or element

However, this doesn't seem to cover the concept of "search".

E.g. in designing a suite of web services that support a Job Search site you might have the following requirements:

  • Get individual Job Advert
    • GET to domain/Job/{id}/
  • Create Job Advert
    • POST to domain/Job/
  • Update Job Advert
    • PUT to domain/Job/
  • Delete Job Advert
    • DELETE to domain/Job/

"Get All Jobs" is also simple:

  • GET to domain/Jobs/

However, how does the job "search" fall into this structure?

You could claim it's a variation of "list collection" and implement as:

  • GET to domain/Jobs/

However, searches can be complex and it's entirely possible to produce a search that generates a long GET string. That is, referencing a SO question here, there are issues using GET strings longer than about 2000 characters.

An example might be in a faceted search - continuing the "job" example.

I may allow for searching on facets - "Technology", "Job Title", "Discipline" as well as free-text keywords, age of job, location and salary.

With a fluid user interface and a large number of technologies and job titles, it is feasible that a search could encompass a large number of facet choices.

Tweak this example to CVs, rather than jobs, bring in even more facets, and you can very easily imagine a search with a hundred facets selected, or even just 40 facets each of which are 50 characters long (e.g. Job Titles, University Names, Employer Names).

In that situation it might be desirable to move a PUT or POST in order to ensure that the search data will get correctly sent. E.g.:

  • POST to domain/Jobs/

But semantically that's an instruction to create a collection.

You could also say you'll express this as the creation of a search:

  • POST to domain/Jobs/Search/

or (as suggested by burninggramma below)

  • POST to domain/JobSearch/

Semantically it may seem to make sense, but you're not actually creating anything, you're making a request for data.

So, semantically it's a GET, but GET isn't guaranteed to support what you need.

So, the question is - Trying to keep as true to RESTful design as possible, whilst ensuring that I'm keeping within the limitations of HTTP, what is the most appropriate design for a search?

share|improve this question
1  
I often intent to use GET domain/Jobs?keyword={keyword}. This works fine for me :) My hope is, that the SEARCH verb will become a standard. programmers.stackexchange.com/questions/233158/… –  Knerd Mar 21 at 11:05
    
Yes, I can see that for a trivial example there isn't a problem. But in the tool we're building it's actually not that unbelievable that we'd end up with a complex search that results in a GET string longer than 2000 characters. What then? –  Rob Baillie Mar 21 at 11:13
    
Actually a very good point. What about specifing a compression technology? –  Knerd Mar 21 at 11:16
2  
GET with a body is allowed by the HTTP spec, may or may not be supported by middleware (sometimes not) ;) and isn't favored as a practice. This comes up on Stackexchange periodically. stackoverflow.com/questions/978061/http-get-with-request-body –  Rob Y Mar 21 at 15:17
1  
I ended up having POST JobSearch create an actual search entity and returning a jobSearchId. Then GET jobs?jobSearch=jobSearchId returns the actual jobs collection. –  Cerad Mar 21 at 15:46

5 Answers 5

up vote 14 down vote accepted

You should not forget that GET requests have some superior advantages over other solutions:

1) GET requests can be copied from the URL bar, they are digested by search engines, they are "friendly". Where "friendly" means that normally a GET request should not modify anything inside your application (idempotent). This is the standard case for a search.

2) All of these concepts are very important not just from user and search engine, but from an architectural, API design standpoint.

3) If you create a workaround with POST/PUT you will have problems which you are not thinking of right now. For example in case of a browser the navigate back button / refresh page / history. These can be solved of course, but that's going to be another workaround, then another and another ...

Considering all this my advice would be:

a) You should be able to fit inside your GET with using clever parameter structure. In extreme case you can even go for tactics like this google search where I set a lot of parameters still its a super short url.

b) Create another entity in your application like JobSearch. Assuming you got so much options, its probable that you will need to store these searches as well and manage them, so its just clearing up your application. You can work with the JobSearch objects as a whole entity, meaning you can test it / use it easier.


Personally I would try to fight with all my claws to get it done with a) and when all hope is lost, I would crawl back with tears in my eyes to option b).

share|improve this answer
    
For clarification, this question is intended to be about web-services design, not web-site design. So whilst the browser behaviour is of interest in the wider scope of the question's interpretation, in the particular case described it's of no consequence. (interesting point though). –  Rob Baillie Mar 21 at 13:02
    
@RobBaillie Ye the browser was just a use case. I wanted to express the fact that your search as a whole is represented by a URL string. Which has a lot of comfort in usability along with other points later in the answer. –  burninggramma Mar 21 at 13:23
    
In point b, is this a simple variation of my own reference to a POST to domain/Jobs/Search/, maybe with domain/JobsSearch/ instead, or did you mean something different? Can you clarify? –  Rob Baillie Mar 21 at 13:31
    
@RobBaillie yes it is, I just don't like nested routes ;] they're evil! –  burninggramma Mar 21 at 13:52
    
I tend to agree. I still think it's a bit of a stretch to state that you are "creating a job search", but I'm worried that I have no other option than to live with that (though probably not actually save it) –  Rob Baillie Mar 21 at 14:19

In REST, resource definition is very broad. It's really however you want to bundle some data.

  • It's useful to think of a search resource as a collection resource. The query parameters, sometimes called the searchable portion of the URI, narrow the resource down to the items the client is interested in.

For example, the main Google URI points to a collection resource of "links to every site on the Internet". Query parameters narrow that to the sites you want to see.

(URI = universal resource identifier, of which URL = universal resource locator, where the familiar "http://" is the default format for a URI. So URL is a locator, but in REST its good to generalize that to a resource identifier. People use them interchangably, though.)

  • Since the resource you're searching on in your example is the jobs collection, it makes sense to search with

GET site/jobs?type=blah&location=here&etc=etc

(return) {jobs: [{job: ...}]}

And then use POST, which is the append or process verb to add new items to that collection:

POST site/jobs

{job: ...}

  • Note that it's the same structure for the job object in each case. A client can GET a collection of jobs, using query params to narrow the search, and then use the same format for one of the items to POST a new job. Or it can take one of those items and PUT to its URI to update that one.

  • For really long or complicated query strings, convention makes it OK to send those as POST requests instead. Bundle the query paramters up as name/value pairs, or nested objects in a JSON or XML structure and send it in the body of the request. For example, if your query has nested data instead of a bunch of name/value pairs. The HTTP spec for POST describes it as the append or process verb. (If you want to sail a battleship through a loophole in REST, use POST.)

I'd use that as the fallback plan, though.

What you lose when you do that though is a) GET is idempotent -- that is, repeatable -- POST is not. So if the call fails, middleware won't automatically retry or cache results, and 2) with the search parameters in the body, you can't cut and paste the URI anymore. That is, the URI isn't a specific identifier for the search you want.

To differntiate between "create" from "search". There are a couple of options that are consistent with REST practice:

  • You could do it in the URI by adding something to the name of the collection, like job-search instead of jobs. That just means you're treating the search collection as a separate resource.

  • Since the semantics of POST is both append OR process, you could identify search bodies with the payload. Like {job: ...} vs. {search: ...}. It's up to the POST logic to post or process it appropriately.

That's pretty much a design/implementation preference. I don't think there's a clear convention.

So, like you've already laid out, the idea is to define a collection resource for jobs

site/jobs

Search with GET + query params to narrow the search. Long or structured data queries go into the body of a POST (possibly to a separate search collection). Create with POST to append to the collection. And update with PUT to a specific URI.

(FWIW the style convention with URIs is to use all lowercase with words separated by hyphens. But that doesn't mean you have to do it that way.)

(Also, I should say that from your question, it's clear that you're a long way down this road. I spelled things out kind of explicitly just to line them up, but your question had already addressed most of the semantic issues in this answer. I was just lacing it with some convention and practice.)

share|improve this answer
    
It's an interesting idea - I wouldn't have considered using the payload to differentiate. It almost seems a little underhand! But I guess the URI scheme doesn't actually contain any verbs - it's the request type that defines the verb. Maybe the payload is closer semantically to the request type than the URI is. The only concern is - Is it transparent to a user of the API? –  Rob Baillie Mar 21 at 15:04
    
In terms of implementation (we're using Node and Express), it may mean the route can't really handle the choice of processing. I'd have to take a look at that... –  Rob Baillie Mar 21 at 15:06
    
I have the same gut feeling, that separating it by URI seems cleaner. I'm kind of going back and forth; it's a judgement call. However, the semantics of HTTP would allow putting it in the body. I like to say, REST is modeled after the World Wide Web, and the WWW was built with GET and POST. –  Rob Y Mar 21 at 15:11

I generally use OData queries, they operate as a GET call but allow you to restrict the properties which are returned and filter them.

You use tokens such as $select= and $filter= so you will end up with a URI which looks something like this:

/users?$select=Id,Name$filter=endswith(Name, 'Smith')

You can also do paging using $skip and $top and ordering.

For more information, check out OData.org. You haven't specified which language you are using, but if it's ASP.NET, the WebApi platform supports OData queries - for others (PHP etc) there are probably libraries you can use to translate them into database queries.

share|improve this answer
3  
An interesting link and worth looking at, but does it solve the fundamental problem described, that GET requests do not support more than 2000 characters in the query string, and it is entirely possible that the query could be far longer than this? –  Rob Baillie Mar 21 at 12:28
    
@RobBaillie I don't think so as it is still a GET call with a query string. I'd suggest using OData wherever you can since it's a standard for querying web data sources and for the few (if any) times the query needs to be so complex that you can't fit it in a 2000 character query, create a specific endpoint which you make a GET call to –  Trevor Pilley Mar 21 at 13:19
    
Can you explain your approach for a "specific endpoint which you make a GET call to"? How might you imagine that endpoint would look? –  Rob Baillie Mar 21 at 13:33
    
@RobBaillie sure - again I'm not sure what tech you are using but in ASP.NET I'd create a specific controller called JobsNearMeAddedInTheLast7Days or whatever to encapsulate the query which is too long/complex for OData and then expose it only via GET calls. –  Trevor Pilley Mar 21 at 13:36
1  
I see. Another interesting thought that probably has some legs, though I'm not sure this would help in my particular case - faceted searching with a lot of facet types and a lot of possible facet values –  Rob Baillie Mar 21 at 14:03

TL;DR: GET for filtering, POST for searching

I make a distinction between filtering the results from listing a collection vs a complex search. The litmus test I use is basically if I need more than filtering (positive, negative, or range) I consider it a more complex search requiring POST.

This tends to be reinforced when thinking about what will be returned. I usually only use GET if a resource has a mostly complete life-cycle (PUT, DELETE, GET, collection GET). Typically in a collection GET I'm returning a list of URIs which are the REST resources that make up that collection. In a complex query I may be pulling from multiple resources in order to construct the response (think SQL join) so I won't be sending back URIs, but actual data. The problem is the data will not be represented in a resource, so I will always have to return data. This seems to me a clear cut case of requiring a POST.

share|improve this answer

One approach to consider is treating the set of possible queries as a collection resource, e.g. /jobs/filters.

POST requests to this resource, with the query parameters in the body, will either create a new resource or identify an existing equivalent filter and return a URL containing its ID: /jobs/filters/12345.

The id can then be used in a GET request for jobs: /jobs?filter=12345. Subsequent GET requests on the filter resource will return the definition of the filter.

This approach has the advantage that it frees you from the query parameter format for filter definition, potentially providing you with more power to define complex filters. OR conditions are one example that I can think of that are difficult to accomplish with query strings.

A drawback to this approach is that you lose readability of the URL (although this can be mitigating by retrieving the definition though a GET request for the filter resource). For this reason you may also want to support the same or a subset of the query parameters on the /jobs resource as you would support for a filter resource. This could be used for shorter queries. If this feature is provided, in order to maintain cacheability between the two types of filtering, when using query parameters on the /jobs resource, the implementation should internally create/reuse a filter resource and return a 302 or 303 status indicating the URL in the form of /jobs?filter=12345.

share|improve this answer
    
My first reaction to this is that whilst it's good information, it's really just a variation on the answer provided by @burninggramma. Essentially it is "create a new entity called filter / search, call to create it and then call to retrieve it". The variation being that the call to retrieve it is more like a call to apply it to a collection. Interesting. However both your and burninggramma's answer suffer from the same problem - I have no desire to create the filters. There will be a huge number of them, and they don't need to be stored except in order to keep a RESTful implementation. –  Rob Baillie Mar 28 at 9:40
1  
Obviously, query parameters are the best solution, but your question specifically asks about how to deal with filter definitions longer than the limit on URLs imposed by some servers. In order to work around length limit you will either need to compress the query string somehow or you need to use a request method which supports specifying a body of arbitrary length. If you don't want to treat filters as a resource, just support a non-restful interface where filter definitions are POSTed. You will lose cacheability, but if you're data is volatile enough it wouldn't benefit from caching anyway. –  pgraham Mar 29 at 14:34

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.