Using natural language grammar in fluent API

Question

I'm tinkering with a query abstraction over WebSQL/Phonegap Database API, and I find myself both drawn to, and doubtful of, defining a fluent API that mimics the use of natural English language grammar.

It might be easiest to explain this via examples. The following are all valid queries in my grammar, and comments explain the intended semantic:

//find user where name equals "foo" or email starts with "foo@"
find("user").where("name").equals("foo").and("email").startsWith("foo@")

//find user where name equals "foo" or "bar"
find("user").where("name").equals("foo").or("bar");

//find user where name equals "foo" or ends with "bar"
find("user").where("name").equals("foo").or().endsWith("bar");

//find user where name equals or ends with "foo"
find("user").where("name").equals().or().endsWith("foo");

//find user where name equals "foo" and email is not like "%contoso.com"
find("user").where("name").equals("foo").and("email").is().not().like("%contoso.com");

//where name is not null
find("user").where("name").is().not().null();

//find post where author is "foo" and id is in (1,2,3)
find("post").where("author").is("foo").and("id").is().in(1, 2, 3);

//find post where id is between 1 and 100
find("post").where("id").is().between(1).and(100);

---

Edit based on Quentin Pradet's feedback: In addition it seems, the API would have to support both plural and singular verb forms, so:

//a equals b
find("post").where("foo").equals(1);

//a and b (both) equal c
find("post").where("foo").and("bar").equal(2);

For the sake of question, let's presume that I haven't exhausted all possible constructs here. Let's also presume that I can cover most correct English sentences - after all, the grammar itself is limited to the verbs and conjuctions defined by SQL.

---

Edit regarding grouping: One "sentence" is one group, and the precedence is as defined in SQL: left to right. Multiple groupings could be expressed with multiple where statements:

//the conjunctive "and()" between where statements is optional
find("post")
  .where("foo").and("bar").equal(2).and()
  .where("baz").isLessThan(5);

---

As you can see, the definition of each method is dependent on the grammatical context it is in. For example the argument to "conjunction methods" or() and and() can either be left out, or refer to a field name or expected value.

To me this feels very intuitive, but I would like you hear your feedback: is this a good, useful API, or should I backpedal to more straighforward implementation?

For the record: this library will also provide a more conventional, non-fluent API based on configuration objects.

Answer 1

I think it's very wrong. I study natural langage and it's full of ambiguity that can only be resolved with context and a lot of human knowledge. The fact that programming languages are not ambiguous is a very good thing! I don't think you want meaning of methods to change according to context:

• This is adds more surprises since you bring ambiguity
• Your users will want to use constructions that you will not have covered, eg. find("user").where("name").and("email").equals("foo");
• It's hard to report errors: what can you do with find("user").where("name").not().is().null();?

---

Let's also presume that I can cover most correct English sentences - after all, the grammar itself is limited to the verbs and conjuctions defined by SQL.

No, you can't cover most correct English sentences. Others have tried before, and it gets very complicated very quickly. It's called Natural language understanding but nobody really tries that: we're trying to solve smaller problems first. For your library, you basically have two options:

• either you restrict yourself to a subset of English: that gives you SQL,
• or you try to cover "English" and you find out that it's not possible due to the ambiguity, complexity and diversity of the language.

Answer 2

I tend to somewhat agree with others' posts that this is not a great design. However, I believe I have different reasons.

You're presenting what I view as a concrete syntax for SQL queries. I strongly believe that concrete syntax can never help a language, only hurt if it is bad.

However, abstract syntax is a different story. Abstract syntax defines the structure of your language and how phrases may be combined to build bigger phrases. I feel that the success of a language strongly depends on the quality of its abstract syntax definition.

My problem with the fluent API isn't that it's ambiguous, or unclear, or not expressive -- it's that it hides the real language and its structure, and in so doing, ends up making things much more complicated than they have to be (by introducing ambiguities, non-obvious syntax errors, etc.).

Since you mentioned that you'll also provide a "more conventional API", it sounds like you already know all this. To that I say "Good!" But that doesn't mean that you can't also develop your fluent API in parallel! A single abstract syntax definition can support multiple concrete syntaxes. While you should keep in mind that the abstract syntax is the real deal, a concrete syntax can also be very helpful.

Answer 3

In addition to Quentin Pradet's very good points, I am doubtful of the alleged benefits of this language.

Presumably the point of a grammar close to natural language is to make it accessible. But SQL already is quite close to natural language. Is one of these really closer to English than the other?

find("user").where("name").equals("foo")

select user from table where name = 'foo'

I don't really see the benefit of your grammar, from a standpoint of intuitiveness or readability. In fact, the SQL version looks more readable (and is easier to type) because of its whitespace.

Answer 4

There are a number of bad less than ideal design decisions that appear to have been made in considering this API.

The first is the question of utility - what purpose does it serve? This appears to be creating a data structure that will compile into a dialect of SQL. Incidentally, the grammar appears to be a limited set of SQL. The question of "what advantage does this serve over just using SQL?" becomes key. If it is more cumbersome to write using the fluent interface than just writing a string with the appropriate interpolation in it then one won't write using this API.

English is ambiguous. Attempting to model a fluent interface on English is a poor choice (you are better off using Latin). When there are multiple possibly valid parsings of the same set of chain of calls, this leads to confusion and surprise. Neither of these are good things to have in an API.

There are more parts to SQL than this API is exposing. Joins (in any of their myriad forms) are notably absent form the example set. Subqueries (foo in (select id from bar)), unions, and group by are a few of the things that are often used. Complex groupings of logic do not appear to be present in any intuitive way.

If one was writing using this API and then found that the API is not capable of expressing the desired query, significant time will be lost. It is a poor choice to be using mixed styles for doing a queries in an application (simple queries in this api, complex in raw sql) - and ultimately the one that is more expressive will be used.

While programming is widespread, English fluency is not. Even with a limitation of the language to "SQL like" there are nuances of how a native speaker would read something and someone who has English as a second or third language.

There is unnecessary redundancy in the API for the sake of English. In particular equal() vs equals() doing the same thing. While I am not certain of it, I believe that is() is a no-op added for sake of a matching English closer. I welcome any to listen to my ranting about redundancy of methods in ruby in chat - don't make the same mistake.

Sit down and write out a comprehensive example set of the queries you want to be able to use. Determine who you will handle all of those examples in a non-ambiguous way that is less cumbersome than the queries themselves. If you can't, consider if it is worthwhile to go down the path of writing the API. SQL is where it is today (it isn't perfect, but I haven't found anything better) over decades of refinement.

RFC 1925 - The Twelve Networking Truths

(12) In protocol design, perfection has been reached not when there is nothing left to add, but when there is nothing left to take away.