Analyzing (un)structured data to convert it into a structured, normalized format.

learn more… | top users | synonyms (1)

1
vote
2answers
109 views

Language/Programming term for paired delimiters [closed]

Can someone help me find the language and/or programming term for delimiters (?) that must be paired? Quotes, parenthesis, angle-brackets, square-brackets, etc. are often used to symbolize these ...
1
vote
2answers
67 views

Passing context around AST nodes

I have various objects inside my AST, such as IfBlock, FunctionBlock, LogicExpression, etc. All of those objects share a context, which is basically a hashmap with some variables. It's a very simple ...
1
vote
2answers
239 views

Lexer/Parser for multidimensional Languages

How does Lexer/Parser work in a 2D programming language like Funciton in order to transform such an unusual source-code to the correct AST?
1
vote
4answers
474 views

Is it possible to create a single tokenizer to parse this?

This extends off this other Q&A thread, but is going into details that are out of scope from the original question. I am generating a parser that is to parse a context-sensitive grammar which can ...
1
vote
2answers
1k views

How to create a Semantic Network like wordnet based on Wikipedia?

I am an undergraduate student and I have to create a Semantic Network based on Wikipedia. This Semantic Network would be similar to Wordnet(except for it is based on Wikipedia and is concerned with ...
1
vote
1answer
272 views

Haskell: Best tools to validate textual input?

In Haskell, there are a few different options to "parsing text". I know of Alex & Happy, Parsec and Attoparsec. Probably some others. I'd like to put together a library where the user can input ...
1
vote
1answer
55 views

File validation rules

I have an application that can accept CSV files to run some operations. The files look like: CREATE USER:username,last_name,first_name,age user1,Smith,John,23 user2,Poppins,Mary,257 There are a ...
1
vote
1answer
84 views

Run a c++ program under lots of different data maps

I want to run a c++ program to process a lot of data from different xml files and output results. I run the program once per file and potentially have around 50 different files. The trouble is each ...
1
vote
1answer
383 views

PHP Simplest XML tokenizer for string processing?

I need to do some text processing tasks with short XML fragments. The section "details" shows an example... My solution is to use a tokenizer based on regular expresions, but it is not elegant and not ...
1
vote
2answers
168 views

How are line/column position data dealt with in parser combinator libraries?

I'm building a parser using a parser combinator library. I need to keep track of where AST nodes started and ended in the textual input -- line and column numbers. How is this problem approached ...
1
vote
1answer
76 views

Writing a parser on top of an XML-based AST: am i doing it right?

I have a sort of AST defined in XML that i'm trying to parse and evaluate. The XML tree contains the tokens and all the information i need. However, i'm finding it difficult to do it "properly". ...
1
vote
1answer
56 views

Can table of content be parsed using some formal grammar?

A table of content can look like: Preface Table of Content Chapter 1 ... 1.1 ... 1.1.1 ... 1.1.2 .... 1.2 ... Summary Exercises Chapter 2 ... ... Appendix ... A ... A.1 ... A.2 ... B ... References ...
1
vote
3answers
501 views

Parsing mathematical expressions with two values that have parentheses and minus signs

I'm trying to parse equations like the ones below, which only have two values or the square root of a certain value, from a text file: 100+100 -100-100 -(100)+(-100) sqrt(100) by the minus ...
1
vote
2answers
296 views

Traversing an AST using Visitors

I'm writing a compiler for a C-like language, and I'm looking for an elegant way to traverse my abstract syntax tree. I'm trying to implement the Visitor pattern, although I'm not convinced that I'm ...
1
vote
1answer
274 views

About AST construction in LL1 non recursive parser

I have implemented a LL1 parser in a non recursive approach with a explicit stack. The following algorithm is from the Dragon Book: set zp to point to the first symbol of w; set X to the top stack ...
1
vote
2answers
291 views

How would my custom language be categorized?

I'm developing my own scripting language to solve some unique challenges for a project. The language takes source code and converts the contents into tokens, and then a command factory is used to ...
1
vote
2answers
118 views

Creating documents from snippets [closed]

From time to time I create whole documents (e.g. XML or proprietary) programmatically. If it easy enough, I just write something like stringbuilder.append("<blah>"); ...
1
vote
2answers
385 views

Should I use a formal grammar for my interpreted scripting language

I have a scripting engine I just published as an open source project. It's been sitting on my harddrive waiting for about a year. My engine of course isn't complete in any way, but it does work for ...
1
vote
2answers
159 views

Creating a text input simplifaction tool

I have been working on a project for several months now to write a web-based tool that will help me at work. I work at a call center and the CRM software that is used is wretched, we basically have to ...
1
vote
2answers
143 views

Extracting useful information from free text

We filter and analyse seats for events. Apparently writing a domain query language for the floor people isn't an option. I'm using C# 4.0 & .NET 4.0, and have relatively free reign to use ...
1
vote
1answer
145 views

How do I parse a header with two different version [ID3] avoiding code duplication?

I really hope you can give me some interesting viewpoints for my situation, because I am not satisfied with my current approach. I am writing an MP3 parser, starting with an ID3v2 parser. Right now ...
1
vote
0answers
119 views

Calculating uncompressed file size without uncompressing file in zlib

I am writing a python program which parses zip (currently only zlib, using DEFLATE compression) files and verifies the correctness of their headers and data. One of the things I'm trying to achieve is ...
1
vote
0answers
663 views

Best practices to parse a log file using Python

I'm writing a Python tool to parse a log file from game server. The log file is of format: ms:classname::id::method::arg1::arg2.... There are a lot of classes, and a lot of methods for each class, ...
1
vote
0answers
66 views

Pre-Compilation Processor:

What I want to do: Parse source code, search for a beginning and closing tag of my own definition (one that does not conflict with any defined patterns in the programming language), and then replace ...
1
vote
1answer
1k views

Using python to parse log files? [closed]

My first useful projects as a programmer has been python scripts that parse out relevant information from log files and do some analysis. I've bumped around and found my way to some functional ...
1
vote
0answers
48 views

Compound container for storing output from a parser

Consider a hierarchical structure like this directory [ [item_0;item_1;...;item_n] [item_0;item_1;inner_directory[...];item_m] other_directory [...] ] ...
1
vote
2answers
209 views

Resources for writing a parser combinator library [closed]

I often use parser combinator libraries, but I've never written one. What are good resources for getting started? In case it matters, I'm using Julia, a functional (but not lazy) language.
1
vote
1answer
262 views

How to read reduce/shift conflicts in LR(1) DFA?

I am reading an explanation (awesome "Parsing Techniques" by D.Grune and C.J.H.Jacobs; p.293 in the 2nd edition) and I moved forward from my last question: How to get lookahead symbol when ...
1
vote
1answer
297 views

Does JAXP natively parse HTML?

So, I whip up a quick test case in Java 7 to grab a couple of elements from random URIs, and see if the built-in parsing stuff will do what I need. Here's the basic setup (with exception handling etc ...
0
votes
3answers
273 views

Do markup languages ​​have the equivalent concept of `semantics` that you can find in C or C++?

Maybe I miss something, but do the so called markup languages have the equivalent concept of semantics that you can find in C or C++? Judging from how you parse the language, you don't really have ...
0
votes
1answer
84 views

Should a structure be refactored into smaller structures?

In one of the projects (in C), I have the code that looks like this (the names are obfuscated specially for question to not give out details of the probject): typedef struct Module_s { /* These ...
0
votes
2answers
212 views

Using a parser to locate faulty code

Lately I've been working a lot in PHP and have run into an abnormally large number of parsing errors. I realize these are my own fault and a result of sloppy initial coding on my part, but it's ...
0
votes
3answers
297 views

Why Double.parseDouble(“ABC”) not returns Double.NaN?

This code: Double.parseDouble("ABC") throws a NumberFormatException. Why is it wrong to expect a Double.NaN (NaN is literally Not-A-Number). A working example is this: public static void ...
0
votes
1answer
143 views

Is it bad to implement a language in other two languages? [closed]

Ok, so I have some understanding about parsers and compilers, at least the basics of how it works, and i've written a calculator and a really small toy language that compiles to another high-level ...
0
votes
1answer
39 views

Rule order for Parsing Lists with LALR(1)

When creating the grammar for parsing a list (something like “ITEM*”) with a LALR(1) parser, this basically can be done in two ways: list : list ITEM | ; or list : ITEM list | ...
0
votes
2answers
208 views

High-level strategy for distinguishing a regular string from invalid JSON (ie. JSON-like string detection)

Disclaimer On Absence of Code: I have no code to post because I haven't started writing; was looking for more theoretical guidance as I doubt I'll have trouble coding it but am pretty befuddled on ...
0
votes
1answer
1k views

Why nginx's http parser doesnt use regular expressions?

I see the http parser written by Igor Sysoev for nginx does not use regular expressions https://github.com/joyent/http-parser What could be the main reason for such design decision? I guess I could ...
0
votes
2answers
158 views

Implementing a first basic interpreter: what should I learn first? [duplicate]

I'm about to implement my own very simple programming language, and an interpreter to execute code in that language. The language will be very basic. Example code: var x = 3 if x > 2 print x if x ...
0
votes
2answers
98 views

Storing tokens during lexing stage

I am currently implementing a lexer that breaks XML files up into tokens, I'm considering ways of passing the tokens onto a parser to create a more useful data structure out of said tokens - my ...
0
votes
2answers
292 views

How to extract operators from the grammar productions for conflict resolution in LALR parser?

Is there some standardized or widely accepted algorithm for picking up operators in shift/reduce conflicts in LALR parser? The question is naive, my problem is not with implementing my solution, but ...
0
votes
2answers
136 views

BNF parsing rule for left associativity

Can someone please assist me with the following question. Write a BNF rule to parse into C -> E C -> E && E C -> E && E && E so that C generates as many E ...
0
votes
2answers
115 views

Large csv's to html report

I am working on a web front end + front end services. I receive good sized csv files (10k lines). My service processes them and condenses them into one larger csv file (up to 300k lines). This ...
0
votes
1answer
145 views

What are known approaches to graphing algebraic expressions?

I am planning to build an expression parser that will be used to graph algebraic functions ( think TI-83 ) with JavaScript. Functions will take the form of f(x)= Aside from typical operators such ...
0
votes
0answers
63 views

Matching groups of similar lines on a generic matching algorithm

I have to write a program to search through a file containing lines and find lines that match to a degree of tolerance but are not necessarily the same. So for example the following lines would match: ...
0
votes
0answers
111 views

Writing Z80 table based assembler/disassembler

I have a long-term project: DIY computer with various processors. One of my wishes not only make hardware, but software too. So I started from assembler/disassembler for Linux, though there is a lot ...
0
votes
1answer
56 views

What is latent parsing in NLP?

I'm reading a paper describing a NLP work, and I hardly catch the concept of a term, "latent parsing". Original paper: http://web.stanford.edu/~angeli/papers/2013-acl-temporal.pdf The figure in the ...
0
votes
0answers
42 views

Dedupe while or after write

I have a summary tool written in python that reads input files, and writes them into a summary file. I have the following stipulations: No duplicates. If it exists, add a count to it. Is it ...
0
votes
0answers
325 views

Parsing Razor-style Templates

I want to build a template engine (ITT not another template engine...) based on Razor. I've been at it for quite a long time not getting anywhere and quite frankly I'm at my limit. I've tried rolling ...
0
votes
1answer
170 views

How lookaheads are propagated in “channel” method of building LALR parser?

The method is described in Dragon Book, however I read about it in ""Parsing Techniques" by D.Grune and C.J.H.Jacobs". I start from my understanding of building channels for NFA: channels are built ...
0
votes
1answer
168 views

Making a sldprt to PDB file converter?

I wanted to create a parser that can read a solidworks file and turn it into a protein data bank file. This has already been done in a program called DiamondCAD. ...