403 reputation
314
bio website
location San Francisco, CA
age
visits member for 3 years, 4 months
seen Jul 9 at 0:22

Aug
21
awarded  Yearling
Jul
2
awarded  Curious
May
15
awarded  Nice Question
Mar
7
comment Data architecture for event log metrics?
UPDATE over a year later we built a system that logged everything, and periodically iterated over the logs counting things, and then stored those counted numbers into a database (could/should have been a a time-series database, but MySQL sufficed). This was a few weeks of work but ended up being a surprisingly powerful/fast approach--when it's just your code iterating over logged JSON, it's easy to add a lot of metadata, and easy for your code to have flexible rules for exactly what it wants to count.
Jan
31
awarded  Popular Question
May
23
asked What's the difference between a stream and a queue?
Jan
8
comment Can Clojure's thread-based agents handle c10k performance?
Thanks! If you are curious, it appears that modern NPTL threads are quite capable, e.g. see http://stackoverflow.com/a/4058620/37610
Jan
8
accepted Can Clojure's thread-based agents handle c10k performance?
Jan
8
comment Data architecture for event log metrics?
@hiwaylon We've ended up using a hybrid system: 1) MySQL where possible (low volume) (makes aggregation easy using SELECT...GROUP BY, can easily store the results of SELECTs), 2) using Graphite for simple large-scale aggregation and visualization, and 3) logging full events for reference, and for watching details of the data flow in in real time. Each has actually been valuable in different ways.
Jan
8
accepted Data architecture for event log metrics?
Nov
17
revised Can Clojure's thread-based agents handle c10k performance?
added 5 characters in body
Nov
17
asked Can Clojure's thread-based agents handle c10k performance?
Oct
4
awarded  Nice Question
Sep
17
comment Data architecture for event log metrics?
"aggregate the data, but store the details in a (compressed) file". Great thought in particular, thanks!
Sep
17
revised Data architecture for event log metrics?
added 214 characters in body
Jul
19
revised Data architecture for event log metrics?
added 23 characters in body
Jul
19
asked Data architecture for event log metrics?
Jul
10
accepted Duck checker in Python: does one exist?
Jul
10
accepted concurrency semantics reading a file being written (ext)
Jul
3
revised concurrency semantics reading a file being written (ext)
added 1 characters in body