New

Allow for multiple CRONS

You would be amazed at the number of issues people have with the single cron.

In my main instance we have over 200 scheduled jobs.  They are a mix of standard, critical and huge jobs.  Some are just plain critical like Inbound EMail and some run a long time like Relationship dedups.  I cant have the Inbound Email backed up behind a job that will take an hour to run.

So I recoded ScheduledJobs to use 3 cron jobs.  I have a General queue, a EMail queue and a Heavy SQL queue and I am just about to add an integration queue for jobs that bring data into Sugar.  This allows the Inbound EMail job to run on time even if a Heavy SQL job is already running.  90% of our jobs are in the general queue.

I am able to designate from the Schedulers screen what queue the job will run in and I am able to flag any job to 'Run Next' meaning it will be the next to be picked (by any queue as run next overrides queue).  

Just to finish it out I added a module that shows the Job_Queue table so our administrators can see whats lined up to run and reorganize it if needed.

Kenneth Brill
Parents

  • I agree with you, we spent a lot of time fine tuning our cron schedule to make sure our time-sensitive jobs did not interfere with the archival, sync, and cleanup jobs while keeping them all running efficiently.

    Your solution sounds quite impressive.

    Our biggest issue right now is we need a more granular prioritization of the Inbound Email mailboxes.

    I have several email addresses that correspond to inbound email boxes, some are for cases, and therefore high priority, others are just for archiving purposes to sync outbound emails to our customer records.

    Ideally one should be able to schedule archival mailboxes to be processed on a different schedule from time sensitive ones. 

    Or at the very least, inbound email boxes that are associated to Cases should be the first to be processed at each run to avoid delays in responding to customers. 

    Granted that with Emails in sidecar we should be using the APIs for archival emails, it would still be nice to have Tech Support emails be processed before Product Feedback, for example.

  • in reply to Francesca Shiekh

    I did that as well, although unrelated to the crons stuff.  We had no control over the order that the inbound email boxes would be read in so I added code to allow us to control the order in which they will be read.  Our next idea is to allow for 2 separate queues for emails so that I can specify that it checks high priority inboxes every 3 minutes and mow priority ones mayube every 1/2 hour.

Comment
  • in reply to Francesca Shiekh

    I did that as well, although unrelated to the crons stuff.  We had no control over the order that the inbound email boxes would be read in so I added code to allow us to control the order in which they will be read.  Our next idea is to allow for 2 separate queues for emails so that I can specify that it checks high priority inboxes every 3 minutes and mow priority ones mayube every 1/2 hour.

Children
No Data