Tell me more ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.
up vote 4 down vote favorite
1

In common life we use text for programming.

But why not gestures or voice? Are there any researches around this topic?

share|improve this question
33  
When I'm programming, I already make enough gestures at the screen. There are usually words involved too. – pdr Sep 5 '12 at 16:56
3  
@AndresF., yes, I'm interested in non-visual programming. – Nikolay Fominyh Sep 5 '12 at 16:59
1  
@DavidKaczynski, they also use special environment to write their code directly to HDD without keyboard. – Nikolay Fominyh Sep 5 '12 at 17:04
1  
As long as you can display the differences from one version to the next. – JeffO Sep 5 '12 at 17:13
2  
@whatsisname Deaf people do this all the time, and they are very proficient at it. – Jarrod Roberson Sep 5 '12 at 19:16
show 12 more comments

closed as not a real question by maple_shaft♦ Sep 5 '12 at 19:28

It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, see the FAQ.

7 Answers

up vote 3 down vote accepted

There's subtext which uses a graphical representation of the code as the code.

It's basically a research project about exactly your question: how can programming be made better by removing it's dependence on a pure-text representation.

share|improve this answer
1  
Interesting read, but putting it in a table/graph/whatnot doesn't necessarily make it easier to understand. It's different, for sure, but better? Eh... – Philip Sep 5 '12 at 17:00
Actually, that's the answer on my question about researches around this topic. – Nikolay Fominyh Sep 8 '12 at 18:42
up vote 11 down vote

There are Many Visual Programming languages. But ultimately they all convert to text.

Programming is not a linear task, how would you edit a Gesture? You could use Speech to Text to code by voice, but again, it's converted to text.

I forgot about Gmail Motion , not really programming but related.

share|improve this answer
May be by converting gesture to visual representation, and then representation to code. – Nikolay Fominyh Sep 5 '12 at 16:53
2  
Technically they all convert to bits not text... This style is quite common in ETL programming. – Chad Sep 5 '12 at 17:12
3  
+1 for "But ultimately they all convert to text" . Although this is not probably what Nikolay had in mind,we could dictate lines of code to the computer thereby using voice to program :P But then again,this would be converted to text. – abhiii5459 Sep 5 '12 at 17:16
up vote 4 down vote

One non-text based language that stands out to me is Piet which uses an image as the code to be compiled.

There's also Scratch which is developed by MIT, designed to make programming more accessibly to beginners and/or children.

share|improve this answer
up vote 3 down vote

Ultimately, a computer program is really just a bunch of ideas that have been strung together to do something useful.

Anything you can use to express an idea can be used to express a program so long as there's an understanding of the mapping between the ideas and their representation. If you're in the forest and all you have to express your concepts are what you find around you, a program could be a bunch of objects lined up like this:

(Red Leaf) (Green Leaf) (Twig) (Stone) (Dead Badger) (Stone) (Yellow Leaf)

Text just happens to be a convenient way to express those ideas because it maps into our (human) language and that language maps into ideas.

share|improve this answer
up vote 3 down vote

Real Life

Non-Visual

Many attempts at general purpose visual programming have been made and all can be empirically be considered failures. They all have some compromise that lets you "script" the graphical elements, because that is the richest, most convenient way to do something. There have been some very domain specific visual only environments that succeed, but they are niche and not general purpose.

Non-visual programming, such as spoken words is a long way off. The closest thing we have today is Siri ( and the like ). But again, that can be considered a domain specific implementation ( search language ).

It is going to be a long time before we get to "Computer; Tea, Earl Grey, Hot".

Non-Tactile

Gestures don't imply programming, they imply non-tactile interaction. There are many examples of this, from the Playstation Eye, Wii to the Kinect. None of them drive a programming environment.

If I close my eyes and touch type and type, as I am doing now; I am doing non-visual "gesture" based programming.

There are many examples of gesture based programming in the music world; if you consider MIDI to be a programming language representation. There is the Theremin which is non-tactile. There is the an abundance of accelerometer based controls in the music world as well. From the "Hot Hand", to iPod/iPhone and iPad driven devices. There are also infrared Theremin style controllers for input as well.

Real Programmers

Real Programmers

Real programmers set the universal constants at the start such that the universe evolves to contain the disk with the data they want.

share|improve this answer
2  
+1 Cause' its funny! HA! Now hes' at a net positive rep, it takes 5 down votes to offset my one up! – Morons Sep 5 '12 at 18:07
up vote 1 down vote

It may sound odd but I think it should be possible very well. For example using music scale you could encode those as symbols to be 'interpreted' by a software. In this way you could give instructions to computer, but as more complex input emerges it could become difficult to represent that.

share|improve this answer
How would you perform a diff between two music-based source files? :P – Andres F. Sep 5 '12 at 20:58
2  
@AndresF.: Two identical waves can cancel each other out if they have the same magnitude but are 180 degrees out of phase...right? so maybe if you change the phase of one "file" by 180 degrees and then play it back the same time as the other file, all you'll hear is silence, and the "diffs" will be the only audible thing. ;) – FrustratedWithFormsDesigner Sep 5 '12 at 21:31
@FrustratedWithFormsDesigner Awesome! – Andres F. Sep 5 '12 at 21:34
I have to come think of this, I guess, while reading GEB. It does weird things with your brain. – AndreasScheinert Sep 7 '12 at 9:24
1  
Apparently you can fly a space ship with a guitar as well with dedicated input controllers! – Jarrod Roberson Sep 10 '12 at 12:00
show 1 more comment
up vote 0 down vote

In the Spanish movie Eva, the computers that control robots are programmed by physically manipulating a complex array of objects and arranging them in specific spatial configurations. There is a library of "parts" from where you can take new objects and add them to the "program". The "object editor" and programming environment is not really made of matter but an augmented reality simulation. Overall, I found the approach very interesting, since the natural properties (size, shape, position) of the physical parts involved map quite well to the values of properties that they may have inside a digital computer.

As far as I can understand, this is all fiction, but not unrealistic. And it doesn't involve any text.

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.