3

I'm currently developing v2.0 of my personal cms. However, I'm stuck. I strongly feel that my old solution for "tasks" is not very dynamic and must be revisited. Currently I'm looking the whole concept of how to dynamically add tasks and run them.

What I mean by task system
Like always in cms software, there are a lot of content, such as images, dynamic files, database entries, etc. I created my own task system aka. The Parser (sounds evil right?) It is made of two files, one parser-classes and other the triggers with the $parser = new ParserClasses(); (I hope you understand what I mean.) Some of the features the parser does:

  • First turn off the site (after the parser is done, turn it back on)
  • Checks for active files (files that don't have parent mysql entry, will be deleted)
  • Clears out old cache
  • Generates new static cache
  • Deals with languages and translation related cache
  • and other tasks related to cms management and maintenance

Why do I want to make it more automatic
Basically all plugins and some themes (add-ons in general,) use this task system. But the problem is that it's not very dynamical, because all functions sit in one huge class. This means that the add-on cannot be installed without opening your php editor.

How I was planning to do it
First I would create my own task language, with preset functions to handle files, cache, etc. (very broad example: GenerateFreshCache('pages')). Now, storing it is the issue.. My ideas are:

  1. To create tasks-table in mysql database. Then add tasks there, with my task language. Later while() them from the database and eval(). (This is a good way, because I can add scheduled times and similar in function, for CRON jobs. Or even categorize the tasks.)
  2. Add .php files to some folder, where they get loaded. They will probably use the same task language, but don't need to be eval()'ed. (I'm scared that this would slow down the parser, because so many files are getting included.)

However, I'm not sure about those ideas and I'm hoping someone has already come across this functionality and can give me pointers. All crazy ideas and concepts are welcome, thanks in advance.

Conclusion
Thanks to everyone who replied. I picked the answer, that helped me the most and not "ohh, too much work, use framework". My task systems core is done and working. Took 2-3 days and currently ~600 lines of code and two classes (outer and inner core.) I finally ended up with two mysql tables, one with tasks (task db) and another with combos (multiple tasks combined together for easier run.) Also, added some features so it would be safe to run via ajax (cant really talk about that, top secret^^). There are might be too many loops, handling the tasks parsing, however this makes adding new tasks very very simple.. Just add it to tasks DB and add the task to cron-list-automatic-run-list or run it straight up via the admin panel.

5
  • I don't fully understand this question. From my understanding your trying to do some sort of "Page level" caching? i.e. page content is held in Db (as in all CMS systems). And then you need to somehow maintain the sync between the database vs the cached static files? If that is the case: you could use something like nginx as full page cache:blog.martinfjordvald.com/2010/09/… I've also used Redis in one of my Custom CMS projects which auto refreshes.
    – Darknight
    Aug 23, 2011 at 13:38
  • @Darknight: Thanks for the comment. The cache itself is not the question/problem, but how to create the most efficient task parser. My cache works on serialized data (local data, not from db) and some other secret methods. My current page loading times are 0.007 average with 1-2 DB queries tops. Aug 23, 2011 at 13:52
  • I've never heard of a "task parser", what exactly is this? any references?
    – Darknight
    Aug 25, 2011 at 22:24
  • wait, is that 7ms or 70ms or 700ms page load? because 7ms would be physically impossible, surely the DNS lookup + network latency would take longer than 7ms?
    – Darknight
    Aug 25, 2011 at 22:27
  • @Darknight: 0.007985 Aug 26, 2011 at 7:19

5 Answers 5

3
+50

I like the idea of storing tasks in the db. I do not like the idea of create a "task language". Why not store everything in the database? Your task table would include an action code and the associated content. When you poll the db for tasks you would have access to all of the content needed to generate the pages. You perform the action based upon the code and can track timestamps of when the actions were taken. This negates your #2 of adding file to a folder.

The generated pages can be stored in the db. Heck you could even translate them on the fly (if they are not HTML already). After you successfully operate on a task you can mark it off as complete. This type of system gives you the ability to re-enable older tasks and doesn't force you you to generate files and store them in the file system (which could be a problem in a hosted environment).

This type of system is used by Wordpress and seem to function just fine.

1
  • Thank you for your answer. It seems that db is the best option. I wanted to create the task language, because it would make some typical tasks easier, like for example the cache related. It would also make things easier for the user. However, thinking about that yet. Basically, everything is being stored in the db anyways, but for pageload related purposes, serialized cache will be created. If i would translate them on the fly, it would add about ~0.03 seconds to my pageload. Cant afford that. But thanks, you confirmed the db idea. Aug 22, 2011 at 14:21
4

Congratulations, you are working on re-inventing the message queue. It is ok, everyone does it at least once. I know I've done it a half dozen times. In any case, rather than inventing a task language and taking on that complexity, what you should look for is a message queueing product for PHP. I would make a recommendation, but I haven't done anything more than monkeypatch in PHP in recent memory so I'm not sure which platform to start with.

If you decide you must do it yourself, I'd avoid writing a task language -- just pass a task type plus data. Instantiate tasks via a factory of some sort, pass in data. Don't forget lots of logging and such so you can have an idea why your task is silently failing.

7
  • 1
    I'm sorry, but this answer does not help me in any way. As I understand, you are bragging on the fact, that I'm reinventing a method, that has done before?! If you have done it before, why not give pointers? I'm aware on the message queuing method, but I tried to made my question little bit more specific and maybe find an alternative on the db idea. You don't need to "do" php to understand the concept.. I'm sorry, if this question seems stupid to you, but I have alot riding on this project.. Aug 22, 2011 at 19:10
  • Sorry you took it that way -- real point was you should not reinvent this wheel but use an existing implementation though I'm not sure which one to use. I fleshed out the answer a bit in any case. Aug 23, 2011 at 13:19
  • Well, I have allergies against other peoples codes. Basically I think I'm dropping out the ask language, but I am going to create this from scratch. Thank you for your edit. Aug 23, 2011 at 13:49
  • 1
    You might want to get treatment for that -- it is very hard to succeed in software without using other people's code. Aug 23, 2011 at 13:52
  • You have a valid point in there, because you must learn from others mistakes. However, so far all my code (php, js, css, templates) are created by myself. When I create everything from scratch, then it is easier to maintain this software in the future. And task parsing system is not hard to make at all. Done it before, but I feel I could make it better, just the visual and concept part is the problem. Aug 23, 2011 at 13:57
1

Why not remove tasks? Make functions which will do things that you do in tasks, and call them when you modify an entity.

So lets say that you are using come translation library to translate news articles.
Why not call generateTranslations when someone updates or creates entity?

EDIT1:(reply to comment) Yeah, but from what i can see, you are fixing mismatched entries in database in task? I think that's totally wrong, you should write your app so such things can't happen. Only thing that MAYBE should be written as task is cache generation, and that is only if you are limited and constrained by some other system. (for example if your server can only generate all cache in one go, but i am fairly sure that is not the case) I re-read your comment, and i can see that you are using task mainly for fixing things, you should really refactor all of your code and fix those bugs. I mean, you are literally patching data to compensate for bugs that you introduced.

EDIT2: Here are some things that should go in tasks: Sending email containing info on some-all articles created since last "same" email was sent.
Parsing content from other sources (getting articles from other sites) IF that source cant notify you of new data that you need to get or send you new data.

2
  • 1
    Good point. However, but call the task system aka. parser as a maintenance system. It will fix all sort of mismatched entries the database, hook pictures to articles, clear cache.. fix cache.. fix permissions etc etc. Also, I have ajax rearrange script for pages order, I don't want to run the cache updater every single rearrange. So basically, the task system is needed. Also, I can set scheduled automated tasks, so in the morning it would clear cache etc. Aug 22, 2011 at 19:06
  • I understand your ideas. Well, the original idea for the parser came from a portal I was developing. And I was testing db's, deleting entries, but their files would still stay on the server. Then the parser would come and crisscross entries with files, files with entries and take care of them. I don't get many bugs or errors to fix. Take the parser as a developer-helper in dev-mode and site-maitainer for the client, when developer leaves the project. Aug 23, 2011 at 4:55
0

I don't see any particular advantage of files over a DB. But why limit yourself to eval(), why not compile and cache the compiled code (if you think a one time hit that can often happen in the background is better in your case)?

0

Each module can register callbacks to occur when you task system runs. Each module will have a file in it that your task system will include before running that will setup the callbacks.

/module.name/task.php

\Task\System::regster('startup', function()
{
    // do something
});

\Task\System::regster('shutdown', function()
{
    // do something
});

/task.php

foreach($modules as $module)
{
    require($module . '/task.php');
}

\Task\System::run();

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.