Feeback for May '09 Developments

Hello,

Here is the feedback of the last month's work.

1.Report Engine :
Open ERP Report Engine has added new type of report called mako2html to parse HTML reports using MAKO template for faster and formatted output. See more detail : http://openerp-team.blogspot.com/2009/05/faster-formating-of-html-reports-using.html


2.Multi-Company :
Now User can directly change the company without having to log out, directly by going through the user preferences. and also Current company is displayed on status bar.


3.Google MAP in Open ERP :
We can see our partner information on google map(name, code, address,...) with icon on map.
Display country wise Partner Turnover, country appears in different colors like
light red color => high turnover
dark red color => low turnover
Display Delivery routes from Warehouse location to Customer location by cities with 10 different colors (by number of delivery to that city from warehouse)

Note : It generates KML file so you can save it on your computer and upload it on google map or google earth.
Required python packages : Google direction package


4.Google Translation in Open ERP :
Now we can translate Open ERP terms / fields / objects etc. in any language using integration of Google Translate. And also we can see unreviewed terms and review easily
see more information : http://www.openerp.com/planet/


5.CRM :
*On Case Section, we can configure the mailgateway through fetchmail to specify the parameters pop address, port, user, password, mailgate script path. When save mailgateway configuration, parameter file will be created per section in "server > bin > .fetchmail > dbname" . and scheduler will be read this file and call system command to fetch mail
*After click on Reminder button in case form, Window will be display message "Email Successfully sent !" if mail successfully sent
*Improve openerp-mailgate.py to accept html body


6.Base Quality module :
This module has been reviewed and should now be more accurate and more usable. As reminder, the base quality module allow you to check the quality of a module using different criteria (pylint, speed efficiency, coding standard followed by openepr, etl...).

It evaluates score in percentage (%) for whole module as well as it can provide score for individual tests, also it can skip some tests if module is not installed in server (workflow test...) so that your score is not bad for no reason.
Use Ponderation value to calculate score. It allows you to save the result of test (Save Report button)
Module also provide generic framework to define your own quality test.
Note : For more information (README.txt)

7.Document modules :

document_rules :
Document Rule module is mainly developed for creating rules on documents.
There are four types of rules which are defined in server actions: To copy document, To move document, To Assign partner to a document and To assign user to a document.
These rules are fired when rules criteria match on the basis of time defined in the scheduler.

document_extention :
Document extension module keeps track of versioning of documents.
Any change is made to the document and saved then it creates version of that document and the versioned document gets locked. The user cannot modify the document if it is versioned. 


8.Usability Improvement :

*Make Partner Category structure view, Products Categories structure view, Analytical Account Structure View same like Stock Locations Structure view
*Rename “Administration > Security > Grant Access to Menus” into "... / Manage Menus"
*Change Order of menus in Product menu. Before, It was like this: Configuration,
Product's by Categories, Pricelists, Products. But Now It is like this: Configuration, Pricelists, Products by Categories, Products
*Rename field "Rentable Product" by "Can be Rent" in Product Form.
Reorder invoicing menus, the New ... Invoice is always the first menu of his level. It should be the latest one. For example, "New Customer Invoice" must be after "Unpaid Customer Invoices".
*On a task form View, second tab, put the text fields.function bellow the one2many and many2one for parent task.
*On a Task View : the fields Importance, Starting Date, Sequence, Partner, can be changed when the task is draft. (can always be changed). Currently, they are readonly.
*In the project management menu, rename "Template of Projects" by "Templates of Projects".
*On a tracking lot, put a link on the right of the lot to get the stock by location.
*On Periodical Inventory, Add the state cancel and the buttons for the following transitions: draft -> cancel, cancel -> draft
*On Outgoing products form, the stock.move lines must have a colour according to their state: grey for confirmed, black for others, blue for done
*The menu: Reporting / Traceability / Stock by Lots should open in list view instead of form view.
*On stock.picking, rename the state: Cancel -> Cancelled
*Now The report dates of inventories displays product field instead of ID of products
*On a purchase order, rename the state "Confirmed" by "Waiting Supplier Ack.".
*On a purchase order form, the “Address” should be renamed to “Supplier Address”. “Partner Ref” has been renamed to “Supplier Ref”.
*In manufacturing, put good icon of the menu Production Orders Planning .
*Change the cases segmentation form, user department form, Scheduler action form to be compliant with v5 guidelines of the form view. (a few fields on the top of the notebook)
*Add Gantt view on CRM Meeting
*Override the name_get method on a stock.location object. If you have 'full':1 in the context, return the full path instead of simply the name: "Tiny Sprl / Stock / Stock Wavre". Change the report Stock by Lots so that it displays the location like this


9.New System in Planning Management :
The module report_analytic_planning has been completely reviewed.
*New integer field for 'Business Days' in planning
*Improve view and do a fixed area in top (as for new design since 5.0)
*Change the default name to be the current month instead of the current day
*Put link between planning and project.task
*New a boolean field on planning line: "Assigned to task(s)" to check task assigned or not for planning line.
*Display Planning By User with him department' user, total of remaining hours of tasks, total plan hours, total open hours, total of Valid holidays Requests between the 2 dates of the planning, calculate free hours, total of timesheet lines between the dates of the planning
*Display Planning By Account with above information


10.ETL :
Improvement
- Pause and restart the job (Under developement)
- Thread server implement on demo
- pickle object to store the current job status

- Python Cprofile
- added concept of cprofile on job in run

New components:

google calendar
- connector => Provide connection to google calendar
- in => fetch events from google calendar
- out => send events to google calendar

google documents
- connector => Provide connection service to google documents
- in => Fetch files from google doc
- out => send files to google doc

gmail contacts
- out => send contacts to gmail contact for given user

Blogger
connector => Provice connection to blogger service
out => send blogs to blogger

XML RPC :

New Test Files
test file for xml_in and xml_out (demo data in xml with different companies):
- added test files which convert xml to dictionary format and vice versa
- try to use XSLT for import and export of xml files (Under development)

etl_interface:
Views:
New components views added(google calendar, google documents)
Dynamic views creation of all components
views improve for job
Wizard:
Add components and transitions from job object
Workflow:
Modified workflow for pause and restart the job


11.Ported etree object instead of minidom object


12.Thurnderbird pluging :
Here are few enhancements, improvements and bugfixing have been done on thunderbird plugin.

* If the thunderbird plugin is not installed or if it's not correctly configured, the proper error message will appear.
* Restructure module addons-extra/thunderbird
* When you want to archive a mail containing an attachment, it opens a window saying "Mail contains attachment!". This window is not useful so removed.
* Rename "Create" button on the bottom left corner to "Create Contact".
* When you archive an email with attachments, it asks Would you like to also save attachments of the email:
- Cancel (don't save anything)
- No (create case only)
- Yes (create case + attachments)
* If it's linked to a partner or address, it will fill in the partner and/or address when it creates a case.
* We have two buttons to archive from the archive window:
* Create Case (the current action)
* Attach (don't create a case but save the email as a normal .eml
file + his attachments, on the selected object). -> this one does not need a section.
* Select all model and search.
* Reviewed all dialogue window. It will ask for Cancel and Ok at the right in each window.
* We have found few bugs and enhancements at the time of testing. We will fix it soon.
Testing on windows platform is going on.
Some modules are in trunk community addons :

1.l10n_nl
Contributor : Veritos - Jan Verlaan (http://www.veritos.nl)

2.printjob
Contributor : Pegueroles SCP (http://www.pegueroles.com)
Featutes :
* Enables batch printing
* Correct memory leak when printing crashes
* Permits reprinting lost PDFs
* Possibility to send jobs to a printer attached to the server

3.price_adjust
Contributor : Niels Huylebroeck (http://www.bubbles-it.be)
Features:
* Price Adjust Wizard :
This wizard will allow you to adjust Cost Price and Sale Price for entire categories.
We can either set a fixed price or use a multiplier to increase or decrease the price.

4.account_customer_ref
Contributor : Alberto Garcia & Tiny
Features :
* Add Sale Customer Ref to Invoice
* In each invoice line show the customer ref of Sale Order origin

5.sale_report_html
Contributor : Tiny (http://www.openerp.com)
Features:
* New HTML report of sale order using MAKO template
* Note: Your system should have mako 0.2.4 installed.

6.account_receipt
Contributor : Paulino
Features:

1.report_truetype
Contributor : EduSense BV (http://www.edusense.nl)

Features :
Module to add any TrueType font to reports. This module transparently extends the report machinery to use TrueType fonts. The normal mechanisms used in Open ERP use the standard Type1 fonts in a latin1 encoding. This prevents usage of wide char character sets, known as UTF-8 and UTF-16 fonts.
This module enables the usage of any TrueType font, which are by definition UTF-8 encoded. The implementation chosen is a one-time registration of all found TrueType fonts in locations as advertised by rl_config.TTFontSearchPath during startup of the server. The path can be adapted to your needs. See reportlab's documentation for the details.
The consequence of this implementation is that new fonts will only be loaded after restarting the Open ERP server. Advantage is that the penalty for scanning and registering is paid only once every server run.
After loading, all reports generated by Open ERP can use these fonts by stating their formal mapped Type1 name, which can be found by running 'ttmkfdir' in the directory containing the fonts (unix like platforms). The typefaces will be listed in the file 'fonts.scale'.

TODO:
* One should be able to replace existing fonts using a map.
* One should know the registered fonts without peeking into the file system, let alone using rare utils.

Open ERP made language translation easy with Google Translate

Hello

Now translate Open ERP terms / fields / objects etc. in any language, as we have made it easier for you with the integration of Google Translate.

We have developed new module which can translate your terms using google translate, You can find menu here: Administration / Translation / Application terms / need review terms…


Google Translate



After Translation



Enjoy using your favorite OpenERP application with Google Translation.

You can get these modules on launchpad in trunk-extra-addon

Name of the module

1. Google_translate

Thanks,

OpenERP Team.

R&D: Feedback of April 09

Well in last month, lots of improvements have been done in the trunk version of Open ERP (server and addons). Here are the most important of it.

Reports:

The Open ERP report engine has been completely rewritten from scratch to speed up and streamline the report generation process. It uses the same technologies than the proceeding engine (RML, ODT) and is API compatible.

Compared to the old engine, a 150 page report for sales orders:

Old report engine: 1 Minute 40 Seconds

New Report Engine: 9 seconds

It also uses much less memory as the XML report is not any more in memory during the rendering process. This allows to generate very big reports without troubles (10.000 pages).

All transitions have been reviewed: odt->rml, rml->pdf, rml->html. We also added a new engine for the generation html->html. This modification is so important that it have been backported to branche/5.

Besides, there is now a new type of report supported: HTML2HTML.

Server:

Since v5 of Open ERP, function fields on objects can be defined as being stored in the database with the store attribute. This has considerably speed up the process, because computations of these fields are done when the user write the record and not any more when it reads the record.

We have now added an extra attribute on store, so that is act like a cache system, and re-evaluate this field, not at each record change but every X minutes. That could be very helpful for stock fields for example that will be automatically updated each hours if they are modified. This allows us to speed up some process (like reading a list of products) by 80% !

Example:
'amount_total': fields.function(_amount_all, method=True, digits=(16,2), string='Total',
store={
'account.invoice': (lambda self, cr, uid, ids, c={}: ids, None, 20, 10),
'account.invoice.tax': (_get_invoice_tax, None, 20, 10),
'account.invoice.line': (_get_invoice_line, None, 20, None),
},
multi='all'),

here amount_total fields will only updated if modified date of account_invoice + given_time(10 hours) <>

eTiny:

Changes made on eTiny:
- speed up (about 7x faster than current stable version)
- replaced kid templates by mako
- removed turbogears dependencies
- use OpenFlashChart 2 -> better graphs
- new widget system

To know more about this, we hereby forward you there

Thanks to Contributor / Partner : ALMACOM

Addons:

• Intra-Stat

  We now have a better tracability of exported goods, thanks to the module report_intrastat that is reporting detail of intrastat entry base on valid and paid invoices. Besides, we also developped a module named electronic_report_intrastat that provides a new wizard to export intrastat entry data into .SDV file format and a track log detail of exporting data.

• Indian Accounting developments

  Some progress have been made in the Indian accounting: see l10n_chart_in (module providing the chart of indian accouting), account_invoice_india, account_reports_india (module providing the 3 most Important Reports for the Indian Accounting: Trial Balance, Profit and Loss Account, Balance Sheet)

• account_base

  Accounting Base Properties is providing the,Basic properties for the Customer / Suppliers for the Accounting i.e. PAN No, TIN No, Sales Tax No, and Exise related Information.(Added wizard which is taking all related detail of partner.)

• multi_company_account

  Convert all related object into multicompany compatible format.

• purchase_number

  * add new feature that will create number on confirmation of the purchase order.

  * modified Format of purchase order report which show the detail purchase order with supplier required detail which are new fields in partner through account_base module. "

• sale_number

  * add new feature that will create number on confirmation of the sale order.

  * modified Format of sale order report which show the detail sale order with customer required detail which are new fields in partner through account_base module.

• Direct Marketing

• CRM Improvements

  Lots of things have been improved. Let's take in example the rules (crm.case.rule) that now offer you more possibilities: you can now make condition on the name of case, or in the communication history. The new system also allows you to trigger a server action as ruling effect.

  We also worked on the ergonomy by defining new wizards and shortcuts to simplify the use, by reviewing the views... the list is too long to post everything here! I encourage you to test and suggest improvemnts through blueprints on launchpad.
  
  

• Thunderbird plugin

  * synchronise all type of attachments.

  * added section field in sync mail window so that we can archive mail to crm.case directly with selected sections.

Webstite:

Also, things have been made on websites:

* all modules are available online for download in all versions (see modules list)

* contributors can write comments in the documentation

* contributors can vote on modules

Community Work:

This section will list all modules that have been added by contributors in addons-extra and addons-community. Again, thanks to everyone that improves Open ERP!

1. location: lp:~openerp-commiter/openobject-addons/trunk-extra-addons

• multilogin_portal

  • contributor: La Ferme du Sart

  • description: Allows xmlrpc queries from partners email+password as:
    - Customer standard user
    - Provider standard user
    Allows xmlrpc queries from computers (IP automatically recognized) as:
    - Computer standard user
    Standard users are set into res.company.

• mrp_bom_customization

  • contributor: Smile

• base_vat_unique

  • contributor: vitrum Network

  • description: Check VAT Number is unique

• l10n_ch_zip_and_bank

  • contributor: Camptocamp SA

  • description: Swiss localisation : Swiss banks and zip codes initialisation

2. location: lp:~openerp-community/openobject-addons/trunk-addons-community

• zarafa

  • contributor: Sednacom

  • description: New objetcs and views to improve use of OpenERP:

    * shortcuts view

    * module view

    * email object
    * Zarafa tools

• product_price_update

  • contributor: Gábor Dukai (http://exploringopenerp.blogspot.com)

  • description: The aim of this module is to allow the automatic update of the price fields of products.
    * added a new pricelist type called 'Internal Pricelist' (currently, we have only 2 pricelist types: Sale and Purchase Pricelist)
    * Created a wizard button in the menu Products>Pricelist called 'Update Product Prices' * After filling in the wizard form and clicking on 'Update', it will change the selected

    price field of all products in the categories that we were selected in the wizard.

• base_report_unicode

  • contributor: Gábor Dukai (http://exploringopenerp.blogspot.com)

  • description: This module replaces the standard PDF Type1 fonts with TrueType fonts that have unicode characters. The module contains the DejaVu fonts v2.29 from http://dejavu-fonts.org/
    With this module you can continue to use the old font names in the templates, they will be replaced with the DejaVu font names every time before creating a pdf.

• hnm_heofax

  • contributor: Heonium (http://www.heonium.com/contributions/developpement/envoyer-des-fax-avec-openerptinyerp.html)

  • description: Add a possibility (wizard) to FAX any documents (PDF/PS/Text) from OpenERP. It get fax number and other informations from OpenERP's partners database. Check the module description for further infos.

• stock_period

  • contributor: Sednacom (http://www.sednacom.fr)

  • description: manage stocks in time period

• alerts

  • contributor: Sednacom (http://www.sednacom.fr)

  • description: Add an alert generator on every ressources of the system.

• rollmanagement

  • contributor: Sednacom (http://www.sednacom.fr)

• pricing_policy

  • contributor: Sednacom (http://www.sednacom.fr)

  • description: Pricing policy module. Install this module to enable a strong link between sale and purchase prices.

• ecotaxe

  • contributor: Sednacom (http://www.sednacom.fr)

  • description: Eco participation' management This module adds the management of 'Eco participation' in TinyERP. 'Eco participation', mostly known as 'Ecotaxe', is a french specificity; its purpose is to finance recycling of products with a high cost of treatment. Install this module if you need to manage 'Eco participation' in your business.
    Module features:
    - amount of 'Eco participation' for product by producer
    - 'Eco participation' management in sale orders, purchases orders and invoices.

• business

  • contributor: Sednacom (http://www.sednacom.fr)

  • description: regroup object for a business

• supplier_product

  • contributor: Sednacom (http://www.sednacom.fr)

  • description: Automatic pricelist for product's supplier

• sednacom_extras

  • contributor: Sednacom (http://www.sednacom.fr)

• catalog

  • contributor: Sednacom (http://www.sednacom.fr)

  • description: Report for catalog

• hr_medical

  • contributor: Sednacom (http://www.sednacom.fr)

  • description: Advanced management for Medical Exam

• product_loan

  • contributor: Smile.fr

• account_bankimport

  • contributor: Tiny / Eddy Boer

  • description: This module can import .asc-files (BRI-layout)

• fleet_manager

  • contributor: Sharoon Thomas

  • description: The module is a vertical for fleet operations management

• sale_simple_pos

  • contributor: Smile.fr

  • description: Very Simplified Point Of Sale; if you are not sure about using the official pos module...

• product_hardware_revision

  • contributor: Smile.fr

  • description: every product can have n revisions and also have a current default revision. Then when receiving a product, in the incoming picking, if a stock.production.lot (eg serial number here) is to be created, then a first line of stock.production.lot.revision is created in the stock.production.lot with values copied from the product default revision. The user can still choose an other revision manually before it gets copied. This is really useful to track the exact hardware of a product with electronic products fro instance.

• base_partner_security

  • contributor: Gábor Dukai (http://exploringopenerp.blogspot.com)

  • description: Adds a restricted field to partner categories. That field value is copied to the partners and partner addresses belonging to those categories. With this, it is easy to restrict access to some partner categories using record rules. Example domain to use with record rules or views: [('restricted','=',False)]

• product_price_decay

  • contributor: Smile.fr

  • description: Product Decay the goal of this module, is to estimate the average price of the remaining product stock using the following assertions: - the remaining stock is based on the invoiced products only (from purchases and sales) - everything happen like if we sale first the more expensive remaining products So it's a bit like FIFO (if prices were always falling it would look like FIFO) but it's certainly not a legal way of valuing your stocks. Actually this module only gives you an indicative price that you could base your sale margin upon, so that: - you always ensure a given margin - you make sure your sale price will follow the market, meaning decay hopefully

• product_tax_incl

  • contributor: Smile.fr

  • description: Product Tax Include Allows to compute the based sale untaxed amount given a taxed amount entry (change the list_price on on_change)

We are thankful to all Open ERP contributors for their efforts and time to make this possible, we expect the same in future, while being mutually beneficial.

Open ERP Web - What's going on?

Most of you, if following the latest development on Launchpad project area, might be aware of the big changes with Open ERP Web Client. We migrated the web client to CherryPy3 droping the TurboGears completely and migrated Kid templates by faster Mako templates as a major step to make the Web Client much more faster and easier to deploy.

All the Kid templates are converted to faster Mako templates, i18n/l18n features have been partially reimplemented using Python Babel, CherryPy2 (TG is built on top of CP2) was replaced with CherryPy3, the latest, much better version of CherryPy Server.

This greatly reduced the pain of getting started with and deploying of Open ERP Web client. Now the number of third party dependencies are reduced to 3-4 pure Python libraries which you can install within the local lib dir with the help of `populate.sh` script, found under the same lib directory. Just get the source from Launchpad, run the populate.sh and launch the web client...

The initial test results are very impressive. We have seen almost 3-5 time speed improvement. Here are the benchmark results of the latest trunk version against the stable 5.0 branch which is running over TurboGears.

The benchmark test was done using Apache Benchmark Tool against relatively bigger Customer Invoice Form view. The command was used like this:

$ ab -C session_id=$session -n 100 "http://localhost:8080/form/edit?model=account.invoice&id=1"

where $session should be replaced with session id (you can use firebug to see the session id).

Here is the comparison between before and after improvements in Open ERP web client

Result of Open ERP Web 5.0 (TurboGears + Kid)	Result of Open ERP Web Trunk (CherryPy3 + Mako)
This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking localhost (be patient).....done Server Software: CherryPy/2.3.0 Server Hostname: localhost Server Port: 8081 Document Path: /form/edit?model=account.invoice&id=1 Document Length: 79965 bytes	This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking localhost (be patient).....done Server Software: CherryPy/3.1.2 Server Hostname: localhost Server Port: 8080 Document Path: /form/edit?model=account.invoice&id=1 Document Length: 90394 bytes
Concurrency Level: 1	Concurrency Level: 1
Time taken for tests: 166.323 seconds	Time taken for tests: 42.054 seconds
Complete requests: 100	Complete requests: 100
Failed requests: 0	Failed requests: 0
Write errors: 0	Write errors: 0
Total transferred: 8022000 bytes	Total transferred: 9063400 bytes
HTML transferred: 7996500 bytes	HTML transferred: 9039400 bytes
Requests per second: 0.60 [#/sec] (mean)	Requests per second: 2.38 [#/sec] (mean)

Time per request: 1663.228 [ms] (mean)	Time per request: 420.543 [ms] (mean)
Time per request: 1663.228 [ms] (mean, across all concurrent requests)	Time per request: 420.543 [ms] (mean, across all concurrent requests)
Transfer rate: 47.10 [Kbytes/sec] received	Transfer rate: 210.47 [Kbytes/sec] received
Connection Times (ms)	Connection Times (ms)
min mean[+/-sd] median max	min mean[+/-sd] median max
Connect: 0 0 0.0 0 0	Connect: 0 0 0.0 0 0
Processing: 1556 1663 71.3 1663 1856	Processing: 382 420 27.7 415 523
Waiting: 1555 1662 71.3 1662 1855	Waiting: 381 420 27.7 415 522
Total: 1556 1663 71.3 1663 1856	Total: 382 420 27.7 416 523
Percentage of the requests served within a certain time (ms)	Percentage of the requests served within a certain time (ms)
50% 1663	50% 416
66% 1681	66% 418

75% 1695	75% 420
80% 1715	80% 424
90% 1775	90% 436
95% 1801	95% 512
98% 1829	98% 520
99% 1856	99% 523
100% 1856 (longest request)	100% 523 (longest request)

Conclusion

You can see significant performance boost in second test result. We observed 3-5 time speedup. There are still room to improve the performance further. Like reducing RPC calls, catching results of some computationally heavy functions. Implementing a way to directly call server methods if web client and server are installed on the same system (completely eliminating TCP layer).

What's next?

Besides this, we also planned to make web client modular implementing plugin system which let you create your own plugins to extend the functionality of the web client. We appreciate your ideas and suggestions in this regards.

Also, we have planned for a better main menu & tabified interface (similar to GTK client) where you always have access to the main menu and all views will be opened in tabs inside the same html page instead of utilising browser tabs. We had discussion about such interface previously, you can search the forum for a very good mockup implementation.

We are also thinking of writing some automated unit tests using CP3 testing API. We already wrote some basic tests to check how to proceed. You can run all tests by launching `run-tests.py` script...

We greatly appreciate you ideas, suggestions in this regards...

Thanks and regards to all contributors

Open ERP Team

Save time/money with Open ERP web client.

OpenERP forum with RSS / Atom feed

Now enjoy the extension of our OpenERP Forum with RSS / Atom feed.

You can choose your favorite discussion / topic on Forum and mark as RSS to get on your desired place.

Or you can choose “Atom” from URL of the page from address bar.

After click on RSS / Atom link, you will get a view like…

So, get updated with Forum as and when you wish, on your demand.