An Expert Diagnosis System for the Benchmarking of SMEs’ Performance Sylvain Delisle1 & Josée St-Pierre2 Institut de recherche sur les PME Laboratoire de recherche sur la performance des entreprises Université du Québec à Trois Rivières 1 : Département de mathématiques et d’informatique 2 : Département des sciences de la gestion Trois-Rivières, Québec, Canada
[email protected] Disponible sur le site du Laboratoire de recherche sur la performance des entreprises (www.uqtr.ca/inrpme/larepe) Section Nos recherches / Sous-section Publications et communications
Business Excellence '03 First International Conference on Performance Measures, Benchmarking and Best Practices in New Economy June 10-13 2003, University of Minho Guimaraes, Portugal
http://www.uqtr.ca/~delisle/
p.2
BEX 2003, Portugal, June 10-13, 2003
Benchmarking for SMEs: Why? ¾ Facing increased competitive pressure due to globalization, as well as increased quality and service requirements from their customers, SME manufacturers must increase their productivity and their competitiveness in order to survive and prosper. prosper ¾ Benchmarking is a way to know how well a SME performs compared to others. Goal: compare one’s practices and results with those of other organizations operating in a similar environment. ¾ Usefulness: identify possible sources of inefficiency in the firm in order to bring about appropriate corrective measures to insure optimal performance. ¾ Although fairly well-known in large enterprises, it is a strategic tool that remains relatively undersued in SMEs.
http://www.uqtr.ca/~delisle/
p.3
BEX 2003, Portugal, June 10-13, 2003
From SMEs to a Benchmarking Tool ¾ SME: a unique enterprise, with limited resources, pursuing very diverse goals, subjected to the constraints of its environment ... ¾ Unicity of the firms and conviction of the impossibility of finding relevant information. ¾ Refusal of entrepreneurs to share perceived confidential strategic information. ¾ Difficulties in formulating strategic orientations beyond day-to-day activities. ¾ So, how to approach SME benchmarking then!? ¾ Our answer: the PDG (Performance, Development, Growth) computerized benchmarking tool. ¾ www.pdg-sme.ca
http://www.uqtr.ca/~delisle/
p.4
BEX 2003, Portugal, June 10-13, 2003
The PDG System: Genesis and Goal ¾ Developed by a multidisciplinary team of researchers (in management, marketing, accounting, finance, human resources, engineering, innovation, information technology), SME entrepreneurs, and economic advisors. ¾ Has been in development / use / fine-tuning for 6 years. Most recent version has been in production for 2 years. ¾ Investment of 2M CAN$. ¾ The goal of the PDG system is to evaluate a SME, from an external perspective and on a comparative basis, in order to produce a diagnosis of its performance and potential, complemented with relevant recommendations. ¾ The PDG benefits to SME entrepreneurs, investors / lenders / bankers, and researchers (via its database).
http://www.uqtr.ca/~delisle/
p.5
BEX 2003, Portugal, June 10-13, 2003
The PDG System: A Bird’s Eye View data and results
Intermediary Partner
Oracle Database
report
report
Lab questionnaire financial data
questionnaire, financial data
report
Entrepreneur (SME)
PDG Expert Diagnosis System
information expertise
Multidisciplinary Team of Human Experts
http://www.uqtr.ca/~delisle/
p.6
BEX 2003, Portugal, June 10-13, 2003
The PDG’s Architecture (a “black-box” expert system)
http://www.uqtr.ca/~delisle/
p.7
BEX 2003, Portugal, June 10-13, 2003
The PDG System: Inputs & Outputs ¾ INPUT:preferred reference group + sections to be filled by owner-manager, human resources manager, controller or general manager, and production manager + complete financial statements of the last 5 years. ¾ Input data is computerized and fed into the PDG system. The evaluation is performed in 3 minutes by contrasting a particular SME with an appropriate reference group of SMEs for which we have already collected relevant data.
¾ OUTPUT (10-page report): 28 management practices
(concerning human resources management, production systems and organization, market development activities, accounting, finance and control tools, effectiveness and vulnerability), 20 results
indicators and 22 general information items are evaluated, leading to 14 recommendations on short term actions the evaluated SME could undertake to improve its overall performance.
http://www.uqtr.ca/~delisle/
p.8
BEX 2003, Portugal, June 10-13, 2003
An Excerpt from the Output Report
Management practices that generate results (4 to 6 for each main business function) Results (2 to 5, depending on the function) are commented upon and analyzed in relation to comparable enterprises operating in a relatively similar environment Comments and recommendations are formulated as to the practices adopted by the SMEs in view of their situation compared to the reference group
http://www.uqtr.ca/~delisle/
p.9
BEX 2003, Portugal, June 10-13, 2003
A Closer Look at some of the Results (1) Evaluated SME 32%
Do you have a board of directors, a management committee or an advisory committee that meets more than twice a % of SMEs year? in the reference group
Evaluated SME (green): yes. Reference group (yellow): 32% said yes.
When production-related decisions are taken, production workers are generally: informed after, informed before, consulted, partners, or mandated? Evaluated SME (red square): informed before. Reference group: partners (average).
http://www.uqtr.ca/~delisle/
p.10
BEX 2003, Portugal, June 10-13, 2003
A Closer Look at some of the Results (2) ¾ Certain financial informations, of historical nature, are presented differently. Trend in profits
8.0%
Reference group
35%
7.0%
30%
6.0%
25%
5.0%
20%
4.0%
15%
3.0%
10%
2.0%
5%
1.0%
0%
Net margin
Gross margin
Evaluated SME 40%
1996
1997
1998
1999
Marge brute - Entreprise
Marge brute - GT
Marge nette - Entreprise
Marge nette - GT
0.0%
http://www.uqtr.ca/~delisle/
p.11
BEX 2003, Portugal, June 10-13, 2003
A Closer Look at some of the Results (3) ¾ A one-page summary on each of the following topic is presented: human resources, production systems, production management, sales/marketing, management and control, effectiveness, and vulnerability. Illustrated with graphical representations. ¾ Example comment on practice evaluation of human resources: “Your human resources management practices are generally more advanced than those of your reference group. You could improve your performance, in particular by implementing participative management to involve employees in the growth of the business.” ¾ Example comment on human resources results: “Your overall effectiveness at managing human resources is comparable to that of your reference group. You should pay attention to why certain managerial jobs have a high rate of voluntary departure, with the objective of lowering hiring and training costs.”
http://www.uqtr.ca/~delisle/
p.12
BEX 2003, Portugal, June 10-13, 2003
The PDG’s 8 Main Steps of Processing (1) 1. 2.
3. 4. 5.
The reference group is created according to a number of criteria usually after several iterations. A number of statistical computations (averages, medians, distributions) related to the reference group are performed within the SAS statistical package, after data from the Oracle database has been fetched. Statistical data is passed from SAS to Excel via an ODBC connection. A bunch of automatic, numerical Excel computations take place at this stage in order to define the relative positioning of the evaluated SME to its reference group. Graphics and various representations associated with the numerical values just computed are then produced.
http://www.uqtr.ca/~delisle/
p.13
BEX 2003, Portugal, June 10-13, 2003
The PDG’s 8 Main Steps of Processing (2) 6. 7.
Textual comments on practices and results are computed, in correlation with the computations completed in the previous steps. An evaluation summary, to appear on the report’s first page, is produced for each of these seven main business functions: i. ii. iii. iv. v. vi. vii.
8.
Human resources Production systems Production management Sales / marketing Management and control Effectiveness Vulnerability
The actual PDG output report, containing the benchmarking results, is then produced and saved on disk. The report is displayed on the screen by the Excel application.
http://www.uqtr.ca/~delisle/
p.14
BEX 2003, Portugal, June 10-13, 2003
The PDG System’s Knowledge ¾ A good deal of multi-domain expertise and informal knowledge engineering was invested into the design of the PDG’s extended questionnaire, as well as into the benchmarking system. ¾ The current version of the PDG system, although not implemented with AI techniques (e.g. knowledge base of rules and facts, inference engine) qualifies as a “blackbox” expert system. system A prototype expert system was developed for a subset of the PDG dealing only with human resources.
¾ The knowledge, information and algorithms it uses allow it to produce outputs that only several human experts in different domains would be able to produce in terms of diagnosis quality, including recommendations. ¾ Over 90% of the reports produced by the PDG system need no modification whatsoever before being returned to the evaluated SME.
http://www.uqtr.ca/~delisle/
p.15
BEX 2003, Portugal, June 10-13, 2003
The SMEs in the PDG’s Database ¾ So far, we have produced more than 600 reports and accumulated in the PDG’s database the evaluation results of 400 different manufacturing SMEs. ¾ These manufacturing SMEs: Are mostly located throughout the province of Québec; Have between 20 and 400 employees; Have between 0.5 to 90M CAN$ in sales; Have been in business for 5 to 125 years; Are of all kinds: small, large, performing, in difficulty, etc. ¾ Other SMEs are from the province of Ontario (Canada) and the USA. Recently, a deal was signed with a French businesspersons group for the production of 100 PDG reports per year.
http://www.uqtr.ca/~delisle/
p.16
BEX 2003, Portugal, June 10-13, 2003
Research and the PDG: An Example ¾ A study was made of 307 Canadian manufacturing SMEs that have used the PDG report, including 49 that have done so more than once. ¾ The aim of that study was to show the importance of benchmarking for manufacturing SMEs in the new business environment and to assess the utility of the PDG report (and, implicitly, of the PDG system) to help businesses to identify management practices that needed to be adjusted. ¾ Our results show that this activity allows these organisations to improve their operational performance confirming the usefulness of benchmarking but also, the value of the recommendations included in the PDG report concerning short-term actions to be undertaken to modify management practices.
http://www.uqtr.ca/~delisle/
p.17
BEX 2003, Portugal, June 10-13, 2003
Conclusion ¾ The PDG proves that if the benchmarking approach is tailored to SMEs’ characteristics, an adequate tool can be devised and used to help SMEs increase their performance. ¾ Benchmarking allows entrepreneurs to break the isolation in which they may be caught by comparing and positioning their SME to others (Monkhouse, 1995). ¾ The use of benchmarking first influences operational performance. It also influences the adoption of best management and production practices. ¾ Identification of the combinations of the most effective «good practices», adapted to the reality of SMEs.
http://www.uqtr.ca/~delisle/
p.18
BEX 2003, Portugal, June 10-13, 2003
Current & Future Work ¾ We will very soon start feeding the PDG system with “international” data, i.e. with data coming from SMEs located in Europe (or other countries). ¾ We are starting a migration to more efficient Web-based technologies (Java), thus allowing various improvements and making a further step toward a fully interactive online version of the PDG system. ¾ We are conducting an in-depth revision of the current PDG system in order to reengineer the system’s architecture from a data warehousing and data mining perspective, still within the context of an Oracle database.
http://www.uqtr.ca/~delisle/
p.19
BEX 2003, Portugal, June 10-13, 2003
P r o fe s s io n n e l A d m in is tr a tio n d e l'e n tr e p ô t, d ic tio n n a ir e d e v a r ia b le s , s é c u r ité , c h a r g e m e n t, e tc . A s s is ta n t E x tr a c tio n d e d o n n é e s
M o d u le 1 E n tre p ô t S a is ie d 'u n q u e s tio n n a ir e
M o d u le 2 A p p lic a t io n w e b d e s q u e s t io n n a ir e s
M o d u le 5 A p p lic a t io n d e g e s t io n d e s d o s s ie r s
C h e r c h e u r / s ta tis tic ie n
U tilis a tio n d e c a lc u ls s ta tis tiq u e s e x te r n e s
A lg o r ith m e s d e tr a n s fo r m a tio n
R a p p o r ts lié s a u x d o s s ie r s
B ase de données d e s r é s u lta ts
S a u v e g a rd e d u ra p p o rt
M o d u le 4 A p p lic a t io n d e p r o d u c t io n d e s ra p p o rts
A ffic h a g e e t c o r r e c tio n d u ra p p o rt A s s is ta n t
A s s is ta n t / P r o fe s s io n n e l
Légende :
U tilis a tio n d e s ta tis tiq u e s e t d a ta m in in g
E x tr a c tio n d e d o n n é e s
B ase de données d e s q u e s tio n n a ir e s
E n tr e p r is e
M o d u le 3 A p p lic a t io n d e c a lc u l s t a t is t iq u e e t d e d a t a m in in g
U tilis a tio n d u code de l'e n tr e p ô t
S a u v e g a r d e d u q u e s tio n n a ir e
A ffic h a g e e t m o d ific a tio n d e s d o s s ie r s
E x tr a c tio n d e d o n n é e s
C h a rg e m e n t d e l'e n tr e p ô t
A p p lic a tio n B ase de données D onnées
P o in t d 'a c c è s a u s y s tè m e A lg o r ith m e s d 'u n e a p p lic a tio n
Im p r e s s io n d u r a p p o r t
