API Users endpoint: getting different number of records for different fields listed on PostData. What is the API query cost algorithm? Is there any endpoint that calculates it?

Hi all!

I'm using SugarCRM API v11_22 and I'm testing the Users module.

If I run different queries I get different results, sometimes even errors. What I've noticed is a mechanism under the hood that calculates the load of the query and depending on that decides how many rows to return despite the page Size. Let me explain in detail:

I have 48 records on the Users table and I'm using this endpoint XXXX.sugarcrm.com/.../filter

CASE 1 

If I try the post data of {"max_num":"10","fields":"id","order_by":"id:ASC"} 

it does the pagination fine.

CASE 2 

If I try to get all the fields:

{"max_num":"10","fields":"id,user_name,user_hash,system_generated_password,pwd_last_changed,authenticate_id,sugar_login,picture,first_name,last_name,full_name,name,is_admin,external_auth_only,receive_notifications,send_email_on_mention,description,date_entered,date_modified,last_login,modified_user_id,modified_by_name,created_by,created_by_name,title,department,phoneXXXXX","order_by":"id:ASC"}

It throws an error: "500 Internal Server Error" without any error message in the body. PS: the list of fields includes all the fields that are exposed by the /metadata endpoint

CASE 3

If I split the list of columns into 2 different requests I get different results:

Request 1: 

{"max_num":"10","fields":"id,user_name,user_hash,system_generated_password,pwd_last_changed,authenticate_id,sugar_login,picture,first_name,last_name,full_name,name,is_admin,external_auth_only,receive_notifications,send_email_on_mention,description,date_entered,date_modified,last_login,modified_user_id,modified_by_name,created_by,created_by_name,title,department,phone_home,phone_mobile,phone_work,phone_other,phone_fax,status,address_street,address_city,address_state,address_country,address_postalcode,usertype,license_type,default_team","order_by":"id:ASC"}

>>I get only 2 pages of data then nextPage =-1 

Request 2:

{"offset":"10","max_num":"10","fields":"id,user_name,user_hash,system_generated_password,pwd_last_changed,authenticate_id,sugar_login,picture,first_name,last_name,full_name,name,is_admin,external_auth_only,receive_notifications,send_email_on_mention,description,date_entered,date_modified,last_login,modified_user_id,modified_by_name,created_by,created_by_name,title,department,phone_home,phone_mobile,phone_work,phone_other,phone_fax,status,address_street,address_city,address_state,address_country,address_postalcode,usertype,license_type,default_team","order_by":"id:ASC"}

>>I get all the pages 

What can I do to have a consistent behavior?

Is there any endpoint that calculates the query cost and decides the number of rows to get per page or anything else?

This is a very tricky behavior for our application and leads us to the wrong results.

Please let me know the resolution or if you have any questions!

Thanks!

Parents
  • That's odd  ,

    Thank you for contacting me privately by message, unfortunately, it would not allow me to respond to your private message.

    At a glance, nothing seems incorrect, so I suspect there might be something incorrect within your system (customisations, upgrade etc).

    If I were you, I would look at any errors within logs (Sugar and web server) to attempt to figure out the actual underlying issue, in combination with reaching out to Sugar support.

    I would also verify passing the same filter array of objects, to: POST /Users/filter/count That endpoint will help you to determine the total record count matching the filter, which you can easily cross-check.

    Finally, if you want all fields returned, you can omit the fields parameter in Sugar's API, which might help you with future-proofing as well if you want all future fields to appear.

    If you do need further help, feel free to reach out on my website and we can arrange something.

    All the best!

    Enrico

    --

    Enrico Simonetti

    Sugar veteran (from 2007)

    www.naonis.tech


    Feel free to reach out for consulting regarding:

    • API Integration and Automation Services
    • Sugar Architecture
    • Sugar Performance Optimisation
    • Sugar Consulting, Best Practices and Technical Training
    • AWS and Sugar Technical Help
    • CTO-as-a-service
    • Solutions-as-a-service
    • and more!

    All active SugarCRM certifications

    Actively working remotely with customers based in APAC and in the United States

  • Hi  Thanks for the quick response!

    Yes, the /count endpoint gives the right count of records. 

    I'm listing all the fields because I need to have also the related objects, that by default are not returned. 

    For instance:

    I have a relationship between the "documents" and "cases". If I query the /Documents endpoint and I don't specify the list of fields the "cases" field will not be returned.

    If I specify only the cases, all the others will not be returned. Thus the only option was to list all the fields. Please let me know if you have another solution in mind.

  • Right! That could be the problem then. Perhaps the system does not return the users without related documents, or cases etc?

    I'd retrieve relationships separately from the records.

    Depending on what you are trying to achieve, there will be other approaches. If you are trying to understand if a new relationship is added or removed from a user, a web hook or logic hook might be an answer.

    If you are looking for all documents related to users, you might go the route of for each user, retrieve all their related documents (if any).

    The system eventually will crash if you attempt to retrieve anything a user is related to, in one API call, perhaps hence why you get the 500 error (or even a timeout in some instances).

    If you need help with architecting and even completing a scalable integration that suits your business needs, you know how to reach me.

    Cheers!

    --

    Enrico Simonetti

    Sugar veteran (from 2007)

    www.naonis.tech


    Feel free to reach out for consulting regarding:

    • API Integration and Automation Services
    • Sugar Architecture
    • Sugar Performance Optimisation
    • Sugar Consulting, Best Practices and Technical Training
    • AWS and Sugar Technical Help
    • CTO-as-a-service
    • Solutions-as-a-service
    • and more!

    All active SugarCRM certifications

    Actively working remotely with customers based in APAC and in the United States

Reply
  • Right! That could be the problem then. Perhaps the system does not return the users without related documents, or cases etc?

    I'd retrieve relationships separately from the records.

    Depending on what you are trying to achieve, there will be other approaches. If you are trying to understand if a new relationship is added or removed from a user, a web hook or logic hook might be an answer.

    If you are looking for all documents related to users, you might go the route of for each user, retrieve all their related documents (if any).

    The system eventually will crash if you attempt to retrieve anything a user is related to, in one API call, perhaps hence why you get the 500 error (or even a timeout in some instances).

    If you need help with architecting and even completing a scalable integration that suits your business needs, you know how to reach me.

    Cheers!

    --

    Enrico Simonetti

    Sugar veteran (from 2007)

    www.naonis.tech


    Feel free to reach out for consulting regarding:

    • API Integration and Automation Services
    • Sugar Architecture
    • Sugar Performance Optimisation
    • Sugar Consulting, Best Practices and Technical Training
    • AWS and Sugar Technical Help
    • CTO-as-a-service
    • Solutions-as-a-service
    • and more!

    All active SugarCRM certifications

    Actively working remotely with customers based in APAC and in the United States

Children
No Data