Analyzing (un)structured data to convert it into a structured, normalized format.

learn more… | top users | synonyms (1)

0
votes
2answers
46 views

Storing tokens during lexing stage

I am currently implementing a lexer that breaks XML files up into tokens, I'm considering ways of passing the tokens onto a parser to create a more useful data structure out of said tokens - my ...
-2
votes
1answer
87 views

What is the right way to parse HTML? [on hold]

I've heard, that parsing HTML using the Cthulhu way is not very good. But what are the right ways to parse HTML? Or is it possible to parse it at all?
0
votes
0answers
39 views

Using JavaScript to find the correct offset in bundled files

I am currently making multiple parsers using PEGjs and have implemented my own partial preprocessor which using a RegExp finds and replaces '#include' directives with the desired files, resulting in a ...
5
votes
2answers
260 views

Slight extension for SQL prepared statements syntax. Need advice

In my database abstraction library I am extending SQL prepared statements syntax to hint a parser with expected literal type. I take it as a very essential improvement, my reasoning you can read here. ...
0
votes
1answer
26 views

Making a sldprt to PDB file converter?

I wanted to create a parser that can read a solidworks file and turn it into a protein data bank file. This has already been done in a program called DiamondCAD. ...
1
vote
1answer
62 views

About AST construction in LL1 non recursive parser

I have implemented a LL1 parser in a non recursive approach with a explicit stack. The following algorithm is from the Dragon Book: set zp to point to the first symbol of w; set X to the top stack ...
4
votes
1answer
103 views

How to line-break an email address?

After some discussion, I have come across a rather complicated situation. Say I intend to display an email address. I have, obviously, limited space available on the screen - be that browser or ...
2
votes
1answer
623 views

Using python to parse log files? [closed]

My first useful projects as a programmer has been python scripts that parse out relevant information from log files and do some analysis. I've bumped around and found my way to some functional ...
0
votes
0answers
74 views

Parsing Razor-style Templates

I want to build a template engine (ITT not another template engine...) based on Razor. I've been at it for quite a long time not getting anywhere and quite frankly I'm at my limit. I've tried rolling ...
0
votes
1answer
52 views

Suggestion on how to fill a web form (several times) [closed]

I need to fill a form using data from a CSV file. I was planning to use CURL+PHP to do it, but then I realized the form has several steps (one on each page), plus it uses javascript to fill hidden ...
1
vote
2answers
137 views

Parser and interpreter knowledge as a way to gauge programmer ability [closed]

This is only anecdotal evidence but from my past encounters with programmers at various workplaces the programmers that understand the fundamentals of parsing and interpreting seem to be overall ...
6
votes
3answers
239 views

Clarification about Grammars , Lexers and Parsers

Background info (May Skip): I am working on a task we have been set at uni in which we have to design a grammar for a DSL we have been provided with. The grammar must be in BNF or EBNF. As well as ...
1
vote
0answers
45 views

Compound container for storing output from a parser

Consider a hierarchical structure like this directory [ [item_0;item_1;...;item_n] [item_0;item_1;inner_directory[...];item_m] other_directory [...] ] ...
1
vote
1answer
141 views

Grammar: start: (a b)? a c; Input: a d. Which error correct at position 2? 1. expected “b”, “c”. OR expected “c”

I am writing a parser (a parser generator for PEGs) and am trying to figure out how to report errors. Grammar: rule: (a b)? a c ; Input: a d Question: Which error message correct at position 2 ...
3
votes
2answers
125 views

Sentence Tree v/s Words List

I was recently tasked with building a Name Entity Recognizer as part of a project. The objective was to parse a given sentence and come up with all the possible combinations of the entities. One ...
0
votes
1answer
77 views

Should a structure be refactored into smaller structures?

In one of the projects (in C), I have the code that looks like this (the names are obfuscated specially for question to not give out details of the probject): typedef struct Module_s { /* These ...
5
votes
1answer
172 views

Does a GPL Bison grammar infect my application?

I am thinking about using a GPL Bison grammar for my own compiler. Will the grammar "infect" my parser such that it needs to be open source? The grammar - in terms of - the input of bison is GPL.
3
votes
1answer
127 views

Object oriented parser model review

First time I make use of Code Review. At the moment I'm creating an Android application and the company I'm developing it for, has a specific file format that can be used to give text based data ...
25
votes
1answer
899 views

The Inglish parser (for The Hobbit 1982)

Was fascinated to read about the text adventure game The Hobbit which featured an incredibly robust parser called "Inglish": ...Inglish allowed one to type advanced sentences such as "ask Gandalf ...
0
votes
1answer
142 views

Haskell: Best tools to validate textual input?

In Haskell, there are a few different options to "parsing text". I know of Alex & Happy, Parsec and Attoparsec. Probably some others. I'd like to put together a library where the user can input ...
2
votes
1answer
132 views

content fingerprint algorithm

I have a database containing just simple URLs. It is as simple as it sounds for now and URL can link to a website or a document(i.e. anything parseable to text). Now I have a simple code which ...
2
votes
2answers
1k views

An ideal way to decode JSON documents in C?

Assuming I have an API to consume that uses JSON as a data transmission method, what is an ideal way to decode the JSON returned by each API resource? For example, in Java I'd create a class for each ...
1
vote
0answers
119 views

Implementing semantic actions in table driven predictive parsing

When doing table driven predictive parsing on an LL(1) grammar, (as explained in good detail here for example), how can we augment the algorithm to allow processing of semantic actions while doing the ...
0
votes
0answers
114 views

DDD (with MS's Silk Project) Where would a Parsing Class go?

I am working on a large scale project that involves a Web Application (done in ASP.NET MVC 4) and several Windows services (C#.NET 4.0). We followed the solution and project layout guidelines from the ...
4
votes
2answers
236 views

Should I commit my generated parser to source control?

I'm using a parser generator to build a compiler. Should I commit the source files produced by the parser generator? I want to commit them to avoid a dependency on the parser generator during the ...
2
votes
4answers
206 views

The right place to embed a parser : Server or the Client?

I'm writing a database management software. It has a client-server architecture where the server has the piece of code to store and retrieve. The client has to get the input from the user, connects to ...
1
vote
2answers
285 views

How would my custom language be categorized?

I'm developing my own scripting language to solve some unique challenges for a project. The language takes source code and converts the contents into tokens, and then a command factory is used to ...
-3
votes
1answer
186 views

Best Object Oriented way of parsing a model with several fields(>50) and with null checks

I have a JSON-based data which contains many fields from a particular model. Any value from the list of fields can be null. I am trying to find the best object oriented way of parsing it and also it ...
3
votes
3answers
212 views

What is the proper way to distinguish between keywords and identifiers?

I'm aware that most modern languages use reserved words to prevent things like keywords from being used as identifiers. Reserved words aside, let's assume a language that allows keywords to be used ...
2
votes
1answer
534 views

Any ASN.1 parsers written completely in managed code & safe from buffer overflows

ASN.1 is a data format used on HTTPS certificates, and various other critical pieces of infrastructure. I need to analyze some potentially hostile ASN.1 data, and want to ensure my code is safe from ...
5
votes
2answers
460 views

Learning YACC nowadays, does it make sense? [closed]

I have a huge project that is using YACC and I would need to fix a bug in it. I might ask someone else who wrote that to fix it but I'm interested in how compilers work. Does it make sense to learn ...
1
vote
1answer
228 views

PHP Simplest XML tokenizer for string processing?

I need to do some text processing tasks with short XML fragments. The section "details" shows an example... My solution is to use a tokenizer based on regular expresions, but it is not elegant and not ...
1
vote
2answers
83 views

Creating documents from snippets

From time to time I create whole documents (e.g. XML or proprietary) programmatically. If it easy enough, I just write something like stringbuilder.append("<blah>"); ...
0
votes
1answer
506 views

Why nginx's http parser doesnt use regular expressions?

I see the http parser written by Igor Sysoev for nginx does not use regular expressions https://github.com/joyent/http-parser What could be the main reason for such design decision? I guess I could ...
12
votes
2answers
735 views

What does scannerless parsing have to do with the “Dangling Else Problem”?

I do not understand this sentence from the Wikipedia article on the Dangling Else problem: [The Dangling Else problem] is a problem that often comes up in compiler construction, especially ...
8
votes
3answers
481 views

What the correct algorithm to invert italics in a mixed text?

The question motivations was depicted in the section below. There are many ways to make text italic, so, perhaps, there are more than one good "swap italics algorithm". The problem reveals some ...
34
votes
6answers
2k views

Why was strict parsing not chosen for HTML?

I am pretty much unaware about the detailed computing history that surrounds internet but it has always made me think why strict parsing was not chosen when creating HTML. Browsers accept any rubbish ...
3
votes
1answer
146 views

Chosing a parser for a code beautifier

I'm in the planning stage of making a code beautifier (similar to AStyle or Uncrustify) - originally I was going to just contribute to one of those projects, but reviewing their source led me to the ...
2
votes
4answers
316 views

Is it possible to create a single tokenizer to parse this?

This extends off this other Q&A thread, but is going into details that are out of scope from the original question. I am generating a parser that is to parse a context-sensitive grammar which can ...
-2
votes
2answers
271 views

What is the name of a grammar which can change its tokenizer in mid parse?

I was creating a language and discovered that my language tokenizer would have to change depending where in the parse it is. I.e. abc[1] would be parsed as 4 tokens (abc, [, 1, ]), where as { abc[1] ...
11
votes
3answers
1k views

Writing a Compiler Compiler - Insight on Use and Features

This is part of a series of questions which focuses on the sister project to the Abstraction Project, which aims to abstract the concepts used in language design in the form of a framework. The ...
0
votes
0answers
68 views

Considerations for beginning work on a unified search

I have become interested in creating a unified search for a corporate asset management database. My goal is to allow users to submit queries like: stuff in building 3210 stuff in building 3210 owned ...
2
votes
2answers
136 views

Resources for writing a parser combinator library [closed]

I often use parser combinator libraries, but I've never written one. What are good resources for getting started? In case it matters, I'm using Julia, a functional (but not lazy) language.
1
vote
2answers
265 views

Should I use a formal grammar for my interpreted scripting language

I have a scripting engine I just published as an open source project. It's been sitting on my harddrive waiting for about a year. My engine of course isn't complete in any way, but it does work for ...
2
votes
1answer
283 views

How to choose a proper parser generator for PHP

Some programmers avoid regexes in some situations (see this popular @nickf comment), perhaps using a parsing framework such as Lex/Yacc. Others prefer to stay within PHP, perhaps using regular ...
13
votes
3answers
2k views

Implementing the Visitor Pattern for an Abstract Syntax Tree

I'm in the process of creating my own programming language, which I do for learning purposes. I already wrote the lexer and a recursive descent parser for a subset of my language (I currently support ...
1
vote
2answers
137 views

How are line/column position data dealt with in parser combinator libraries?

I'm building a parser using a parser combinator library. I need to keep track of where AST nodes started and ended in the textual input -- line and column numbers. How is this problem approached ...
8
votes
3answers
4k views

Generic file parser design in Java using the Strategy pattern

I am working on a product in which the responsibility of one of the modules is to parse XML files and dump the required content in a database. Even though the present requirement is only to parse XML ...
1
vote
2answers
150 views

Creating a text input simplifaction tool

I have been working on a project for several months now to write a web-based tool that will help me at work. I work at a call center and the CRM software that is used is wretched, we basically have to ...
2
votes
1answer
194 views

Practical reference for learning about graph reduction

Are there any practical references (with actual examples) for getting started implementing a small, lazy functional programming language with graph reduction? A reference that included the lexing and ...