What are the files in Sugar 11 that are called orderMapping.php?

& -

Sugar 11 has added a bunch of new files in the custom directory with the name orderMapping.php. Can you explain what these files are used for? Do they get rebuilt via QRR? 

They causing havoc with Source Code Control.

Example custom/Extension/modules/Opportunities/Ext/LogicHooks/orderMapping.php

<?php
// created: 2021-03-26 10:56:52
$extensionOrderMap = array (
  'modules/Opportunities/Ext/LogicHooks/DeleteOpportunity.php' =>
  array (
    'md5' => '38db1da780d6b18f38b7cb5472758c17',
    'mtime' => 1612792956,
    'is_override' => false,
  ),
  'modules/Opportunities/Ext/LogicHooks/FixWorksheetAccountAssignment.php' =>
  array (
    'md5' => '9aa1f6f0d6be9066c4f142f9af8db4ea',
    'mtime' => 1612792956,
    'is_override' => false,
  ),
  'modules/Opportunities/Ext/LogicHooks/GenerateRenewalOpportunity.php' =>
  array (
    'md5' => '207fa6307d110a83eba589f7ed82082d',
    'mtime' => 1612792956,
    'is_override' => false,
  ),
  'modules/Opportunities/Ext/LogicHooks/OpportunitySalesStatus.php' =>
  array (
    'md5' => 'c35c3364902925e5a15e2cc82e38062e',
    'mtime' => 1612792956,
    'is_override' => false,
  ),
  'modules/Opportunities/Ext/LogicHooks/OpportunitySyncWorksheet.php' =>
  array (
    'md5' => '97276a801d4a0c4d0b40e0dca02a6ca4',
    'mtime' => 1612792956,
    'is_override' => false,
  ),
  'modules/Opportunities/Ext/LogicHooks/QueuePurchaseGeneration.php' =>
  array (
    'md5' => 'de3fe02382aacc28473b01b831b74814',
    'mtime' => 1612792956,
    'is_override' => false,
  ),
  'modules/Opportunities/Ext/LogicHooks/SetCommitStageForClosedWon.php' =>
  array (
    'md5' => 'bf9f44d5d332a045ceeddb7c2557ed14',
    'mtime' => 1612792956,
    'is_override' => false,
  ),
  'modules/Opportunities/Ext/LogicHooks/SetForecastCommitStage.php' =>
  array (
    'md5' => '5c1669e98c87c034c284be74b061e5bf',
    'mtime' => 1612792956,
    'is_override' => false,
  ),
  'modules/Opportunities/Ext/LogicHooks/SyncBestWorstWithLikely.php' =>
  array (
    'md5' => 'cff052a0a6215992ef2e9004f9983c36',
    'mtime' => 1612792956,
    'is_override' => false,
  ),
);

Parents
  • The explanation on the page https://sugarclub.sugarcrm.com/dev-club/w/dev-tutorials/414/sugar-11-0-q2-2021-customization-guide; is unclear. Can you please provide full details? It appears that the files are only referenced in the ModuleInstall/ModuleInstaller.php

  • ,

    This functionality was developed as part of a bug fix for #72177. In some conditions (especially when conflicting customisations were developed) there was the risk that the merging order of files might change when an instance's file system is copied/moved, depending on the type of file system used.

    These new files store the order of the first time the customisation file is found (and should not be messed with), so that going forward, the customisations are merged in the order on which are first encountered, independently from the file timestamp.

    My take is that you should version those files, so that your system's merging consistency is kept intact

    Cheers

    --

    Enrico Simonetti

    Sugar veteran (from 2007)

    www.naonis.tech


    Feel free to reach out for consulting regarding:

    • API Integration and Automation Services
    • Sugar Architecture
    • Sugar Performance Optimisation
    • Sugar Consulting, Best Practices and Technical Training
    • AWS and Sugar Technical Help
    • CTO-as-a-service
    • Solutions-as-a-service
    • and more!

    All active SugarCRM certifications

    Actively working remotely with customers based in APAC and in the United States

  •  the bug you have identified, https://portal.sugarondemand.com/#supp_Bugs/6875dd3f-76c8-f5da-9329-54c0a5568bf1, was flagged in 2015. The issue with the order has been in place since 2004 (founding of SugarCRM). Adding orderMapping should have been a major discussion at both the PRB (Product Review Board) and the Developers webinar!

    The general workflow would be to build customization in dev. The orderMapping would then be updated in dev. The new orderMapping is now in conflict with both UAT and PROD. 

    What is your advice on the development workflow with this new file? When we build a package, what is your advice for including the orderMapping file? Does the Module Loader flag this file and remove it? Does Module Loader indicate that there could be a conflict? Does uninstalling a package via Module Loader remove this file, update the file?

  • ,

    These are auto-generated files which you should not modify manually nor include into module loadable packages in any way.

    I am a little confused on why you would have conflicts. Are you mixing the full versioning of your code base with Studio/Module Loadable Packages changes by any chance? Or how are you working today?

    As you would understand there are two approaches based on Module Loadable Packages (cloud or on-site) and full system versioning (on-site).

    These are the approaches I see going forward:

    • Sugar Cloud: you should not worry about their existence, they are there to help the system have a fixed order of merging that will not change. New files added, will be merged last. Never include those files into Module Loadable Packages, modify them, nor delete them
    • On site: you should version them as part of your code tree as those files will evolve automatically (if today your approach is to version the full code or even just the custom directory). These files should not be modified or deleted manually. If instead your approach is to just use Module Loadable Packages, you would follow the same approach as Sugar Cloud above

    When copying/promoting an instance (eg: from Dev to UAT), you should copy those files as any other file across, and not modify them manually in any way.

    They are really a system's reference point on when a file was first seen/merged, so that the order of merge is consistent. They are not supposed to be changed manually.

    Hope that answers your questions

    --

    Enrico Simonetti

    Sugar veteran (from 2007)

    www.naonis.tech


    Feel free to reach out for consulting regarding:

    • API Integration and Automation Services
    • Sugar Architecture
    • Sugar Performance Optimisation
    • Sugar Consulting, Best Practices and Technical Training
    • AWS and Sugar Technical Help
    • CTO-as-a-service
    • Solutions-as-a-service
    • and more!

    All active SugarCRM certifications

    Actively working remotely with customers based in APAC and in the United States

  • I have to agree here with Jeff why wasn't this discussed in great detail other then just a blogpost? We for instance make a release that contains ALL the customisations of a specific instance and we will install that big release whenever we do a project. Now that I know that I need to exclude these files from the release I will to so but why are these files in the custom directory in the first place? Shouldn't these be in the database or somewhere else but certainly not in my opinion, in the custom directory which is a place for partners to play around in. 

    What happens if we do a release and install the version that we have locally over the those on the cloud? 

  • there is a HUGE issue. Every time a QRR is done. the orderMapping.php files are updated. The date at the top is updated. If there are no changes in the orderMapping.php the files should not be updated. 

    I have conflicts all over the place.

    local dev

    client dev

    client uat

    client production

    I have no idea which files are correct or incorrect. There are a few thousand orderMapping.php files that are updated with each QRR. 

    When doing a QRR; the date should not be part of the file. Can you remove the date? Then only if the data in the files change would the files be different!!!!

  • Or instead of having the files all over the custom directory move them to a separate directory? Or even store the data in the database?

  • Hi Jeroen, 

    We have provided guidance in past that you should not be versioning generated files within source control and not something you'd want to include in Module Loadable Packages either.

    1) Generated files are derivative of code or changes you've made somewhere else. By including generated files in your package or source control, you allow for possibility of making the mistake of changing the generated file and not the source. This means your changes get overwritten

    2) They unnecessarily increase the size of your package.

    3) In particular, if your Extensions are installed into an environment with existing Extensions, then your generated Extension files will not include those other extensions until a QRR is run and the extensions are completely rebuilt.

    Some references:

    Sugar Developer Guide section on gitignore: https://support.sugarcrm.com/Documentation/Sugar_Developer/Sugar_Developer_Guide_11.0/Introduction/Development_Methodology/#Using_gitignore_Files

    How to write code for SugarCloud webinar: https://sugarclub.sugarcrm.com/dev-club/m/event-recaps/864

    App Ecosystem @ SugarCRM

Reply Children
  • I was aware of that we need to ignore certain files. the .ext.php files etc but I wasn't aware that these mappng files where introduced. When I check the link for the gitignore I don't see any mention of these files by the way. So you can change the documentation to add them to the default gitignore file? 

  • If I use the .gitignore you're providing (and if I'm reading correctly), those ordermapping files will be versioned, but you're telling use they shouldn't be, I guess we can agree on that being ambiguous? :)

    In general, I wonder,what do you call "generated files"? As far as I know, every time someone creates a field, or does basically anything in the studio, some files are generated (vardefs, view files, dependencies...), but we should indeed be versioning those. I'm being picky, but newer developers will be confused.

    There is a lot of confusion around the subject of versioning with this product, that's because the knowledge that some generated files should be versioned and some shouldn't only comes from trial and error, and that's not ok at all. We have a perfect example here : a new type of file appears in a new version, no one knows what to do with it. We had the same issue with role-based view files back then if I remember correctly : they are views so they should be versioned, but they're named after a technical ID that is environment-specific by nature... We figured it out, but it wasn't the best experience ever, to say the least.

    Any product or framework should provide a default .gitignore that doesn't need too much tweaking, default git hooks if needed, and developers (especially newer ones) shouldn't have to worry about making a mess in their repository by clicking a button.

    But anyway, good to know we can ignore those files, but I wonder how my client's instance will know about file orders if that order has been decided on my local instance, and the order files haven't been packaged or versioned? Will that be an issue?

  • Hi Chris,

    The .gitignore provided in developer guide was intended to just be an example. But I can make sure it gets updated to properly exclude generated orderMapping.php files.

    I think it's helpful development model to work from a Module Loadable Package like we've done with Professor M.

    https://github.com/sugarcrm/school

    That keeps your source code pristine and separate from the Sugar instance. It's also necessary anyway when developing code for SugarCloud.

    App Ecosystem @ SugarCRM

  • We have provided guidance in past that you should not be versioning generated files within source control and not something you'd want to include in Module Loadable Packages either.

    Yes true. But the only mention of the files are in the release notes. There are no mentions on where these files are stored and how they work. Thats what I was looking for. If we don't have to add them to a module loadable package that's fine but just let the documentation be clear about that its a generated file and should be treated as such..