NLINK provides several different ways of communicating with other systems. Depending on how a meta-database is configured and how various External Systems (e.g. SAP, SQL Server or any other system NLINK needs to interact with) are set up, the net throughput of NLINK can be greatly affected. This document provides some guidelines on troubleshooting performance related issues in an NLINK deployment.
Software can rarely be expected to exist in isolation. In a typical environment, it has to co-exist with other software from disparate vendors. Given the wide choice of hardware available, it can be difficult to comprehensively indicate how to performance-tune a specific software application. When trying to figure out performance issues, the basic questions that need to be considered are:
- Is hardware causing a bottleneck?
- Which processes are consuming resources?
- Is NLINK overloading the machine?
- Is the bottleneck some other external system (that NLINK is configured to interact with)?
- Are there excessive errors (or warnings) generated by NLINK?
- Are there any errors or issues that are logged in either the System event log or relevant Application event logs?
The hardware configuration that NLINK runs under can affect its performance for better or worse. For details on the minimum hardware requirements see “NLINK Hardware Requrements”. If your hardware does not meet at least the minimum requirements needed by NLINK, you should consider upgrading the hardware before addressing any performance issues
Microsoft Windows Operating Systems (2008, 2012 R2) comes with a tool called “Performance Monitor”. Performance Monitor can be used to monitor the NLINK Server. See How to Collect Performance Statistics for tips on configuring Performance Monitor. Make sure the sampling rate is set properly so that a long duration of data is captured (over hours or days rathers than minutes or seconds). Various metrics gathered from Performance Monitor can be used to see where maximum resources and time are being spent and understand bottlenecks in systems.
Virtual Bytes
At some point after startup, once all the interfaces reach steady state, the NLINK virtual bytes will fluctuate within a certain range. The period of time the NLINK Server needs to reach steady state will vary in each deployment. The important thing is that the virtual bytes should not increase forever. The practical limit for NLINK virtual bytes is 3 GB on 64-bit Windows, or 2 GB in 32-bit Windows with /3GB switch, or 1.5 GB in 32-bit Windows.
If NLINK's virtual bytes value is consistently increasing, it probably indicates some sort of memory resource issue, which will require further analysis.
- Private Bytes
Similar to virtual bytes, private bytes should reach a steady state value and then should fluctuate within a certain range. The practical limit for NLINK private bytes is 2.5 GB on 64-bit Windows, or 1.5 GB in 32-bit Windows with /3GB switch, or 1 GB in 32-bit Windows.
If NLINK's private bytes value is consistently increasing, it probably indicates some sort of memory resource issue, which will require further analysis.
Processor Time
Processor time can provide a rough indication of throughput and help identify potential issues. if the ”Processor Time” for NLINK is a fraction of the overall time taken to run an Interace, then you should focus on monitoring the other External Systems (e.g. SAP, SQL Server or any other system NLINK needs to interact with) to get a better understanding of the throughput issues.
Disk space
Typical NLINK operation requires about 1 GB of free disk space. In most cases NLINK configuration files take up less than 30 MB. The rest of the disk space requirement depends on the use of advanced features, such as Store and Forward and the amount of logging configured in meta-database.
Most run time logs that NLINK generates can be cleared periodically or backed up to a secondary server as needed. The NLINK trace file 'NLINK.trc" cannot be deleted or moved when NLINK is running.
- Try to set up an environment as close as possible to that expected in production. The amount of RAM and the number and type of other processes running will make a difference, so ensure the hardware is comparable.
- Gather the NLINK statistics over a reasonably long period of time to avoid basing your analysis on short-term fluctuations.
- Try to process data sets that are close to those typically expected in production environment.
- Ensure that all external systems (e.g. SAP, SQL server) that NLINK is configured to interact with are running at expected load levels.
- If you start to notice a period of sluggish NLINK performance the first thing to do is to use the Windows Task Manager application to determine how much CPU and memory the NLINK Server is consuming. You should also review how much CPU and memory any other applications are consuming during this period. Often, sluggish NLINK performance can be a result of another application hogging the available system resources.
- If there is no apparent lack of system resources and NLINK is still behaving below operational expectations the simplest thing to do may be to stop and then re-start the NLINK Server. Note that you should do this using the NLINK Management Module (NMM) or from Windows service control manager (SCM). In either case use the Windows Task Manager to ensure that all NLINK processes are actually stopped before restarting NLINK.
- If you have re-started NLINK and still seen no improvement in performance the next easiest thing to do is to reboot the hardware that NLINK is running on.
- If the NLINK logs folder is rapidly taking up more and more disk space then make sure NLINK is not running in debug mode. The debug mode should be only used for short periods of time for troubleshooting issues and should never be enabled during production.
- NLINK is a multi-threaded application. Multi-threaded applications allow for concurrency, but it is also very challenging to troubleshoot performance issues in multi-threaded applications. The NLINK meta-database can be configured to allow multiple Interfaces to run at same time. Each of the Interfaces runs in its own thread. It is best to isolate the Interface of interest and troubleshoot only that Interface. Make sure (by temporarily modifying the meta-database, if need be) that no other Interfaces are running during the period of monitoring or troubleshooting any one specific interface.
NLINK Server uses the Windows Event log to report any issues/problems encountered at run time. Depending on the volume of information being captured, you will need to set the size of the Event Log accordingly, so that all the events that occur over the monitoring time are capured for analysis. Note that setting Event log to be too big makes it difficult to use. A good guideline would be to set it on the order of 100 MBs (not GBs).
The NLINK Event Log in conjunction with “Performance Monitor” can be used to determine acitivity in NLINK.
- Interface Log files
Using the NLINK Log Message Action, data specific to an Interface can be written to Interface Log files. These files can contain information relevant to specific Interface. If the meta-database is configured to generate Interface Logs then make sure there is enough information in the messages to capture the key issues. These files will be different for each NLINK installation since they are purely configuration-driven.
- NLINK Logs and Trace files
NLINK CoNNectors and eXtenders can generate traces of all their activity. These traces along with Event Log can provide insight into all the activity in NLINK. Some of the contents of Trace files might be useful only to Junot Systems support.Use the NLINK Management Module (NMM) to activate the NLINK traces. The screen is under the Tools >> Settings menu option. Click on the Advanced Options tab to get to the Activate NLINK Server Trace flag. These trace files are different than those generated using NLINK System Properties Debug Level and Create Dump File.
Various metrics gathered from Performance Monitor can be used to see where maximum resources and time are being spent and understand bottlenecks in systems.
A low number (over a long period of time) will indicate that application(s) are starved for memory.If this value is less than 5 percent of the total physical RAM, that means there is insufficient memory, and that can increase paging activity. Adding more physical memory (RAM) might help performance or moving some applications to another box might also be a factor to consider.
Page Faults / Second
The page faults / second counter indicates the frequency of hardware page faults--pages which had to be retrieved from hard disk since they were not in working memory. In a typical system there will be some page faults but this should be a low number. A large number of page faults/sec indicates excessive paging.
Virtual Bytes
At some point after startup, once all the interfaces reach steady state, the NLINK virtual bytes will fluctuate within a certain range. The period of time the NLINK Server needs to reach steady state will vary in each deployment. The important thing is that the virtual bytes should not increase forever. The practical limit for NLINK virtual bytes is 3 GB on 64-bit Windows, or 2 GB in 32-bit Windows with /3GB switch, or 1.5 GB in 32-bit Windows.
If NLINK's virtual bytes value is consistently increasing, it probably indicates some sort of memory resource issue, which will require further analysis.
Similar to virtual bytes, private bytes should reach a steady state value and then should fluctuate within a certain range. The practical limit for NLINK private bytes is 2.5 GB on 64-bit Windows, or 1.5 GB in 32-bit Windows with /3GB switch, or 1 GB in 32-bit Windows.
If NLINK's private bytes value is consistently increasing, it probably indicates some sort of memory resource issue, which will require further analysis.
Processor Time
Processor time can provide a rough indication of throughput and help identify potential issues. if the ”Processor Time” for NLINK is a fraction of the overall time taken to run an Interace, then you should focus on monitoring the other External Systems (e.g. SAP, SQL Server or any other system NLINK needs to interact with) to get a better understanding of the throughput issues.
Disk space
Typical NLINK operation requires about 1 GB of free disk space. In most cases NLINK configuration files take up less than 30 MB. The rest of the disk space requirement depends on the use of advanced features, such as Store and Forward and the amount of logging configured in meta-database.
Most run time logs that NLINK generates can be cleared periodically or backed up to a secondary server as needed. The NLINK trace file 'NLINK.trc" cannot be deleted or moved when NLINK is running.