Test Case

Show / Hide direct link   <MAIN> 2086PA: Wireless Reliability/Integration/Performance test - PP-80367



Version 2
Created on 29/04/2009 17:37:44  by Muralidharan Seshadri
Summary
Preconditions
Status:   New
Priority:   Medium
BASIC INFORMATION
Section:S/W Test Require attachment:No
Team:Network Group:Integration
Lessons Learned:No Phase:SIV, (BBFV), SIT
Locations:Inhouse: AWS and Cambridge, Inhouse: AWS, Inhouse: Cambridge, Compal, Flextronics, Quanta, Wistron Owned By:Eiji Ogata/LENOVO
Objective:To verify the wireless reliability and compatibility functions
CRITERIA
All the wireless functions should work normally.
No performance degradation in a integrated environment.
EUT QUANTITY
minimum of Two
EQUIPMENT
PPA Networking facilities.
Critical Attribute:[0] : WLS - Wireless

[1] : MDM - Modem

[2] : MEM - Memory (Total size)

[3] : ETN - Ethernet

CONFIGURATION
SYSTEM
ITEM
CONFIG-1
CONFIG-2
CONFIG-3
SYSTEM
- PCI Bus (CPU)
Don't care
Don't care
Don't care
- Memory
Max
Max
Max
Adapter
Use all supported WL adapter
Use all supported WL adapter
Use all supported WL adapter
EDC*1
Yes
Yes
Yes
MDC*1
--
--
--
BDC*1
Yes
Yes
Yes
DOCKING
Standalone
- KBD/MOUSE
Don't care
Don't care
Don't care
- CRT
Don't care
Don't care
Don't care
- Bay
DVD drive
NA
NA
- USB
Yes (Mouse)
Yes (Mouse)
Yes (Mouse)

Note : *1 Check for product support

REFERENCE PROCEDURE
Currently PA workshop has multiple Access points


*1 : For details of these Access point/Gateway - Contact System Administrator
STATISTIC INFO
Operating System Win81-64bit Win81-32bit Win10-32bit Win10-64bit Win7- 64bit Win7- 32bit Total
Test Points
Duration
Workload
# Step actions Win81-64bitWin81-32bitWin10-32bitWin10-64bitWin7- 64bitWin7- 32bit
1 Wireless compatibility / Integration test AAAAAA
2 Repeatability AAAAAA
3 Idling Test AAAAAA
4 Reliability Test AAAAAA
5 LongRun AAAAAA
DETAILS
ATTENTION!! Please follow below information.
1. Please check Error and Warning message in Event Viewer before you start the test case and after you complete the test case, and report it to your team lead, then arrange to open defect, if needed.

2. Perform All Test in Battery Mode (Some of the test items need to perform in AC Mode, follow each Test Case).

3. All Test Activity should be performed with AMT Enabled as Default (applicable on vPro enabled systems).

4. Please check Usability / Cosmetic issues (correct sentence/appropriate display/etc.) and open defect, if needed.

Wireless compatibility / Integration test
Test Item Config
Result
Win 8
Win-7
Win-8
Modem/Wireless capability
Configure wireless
Configure modem
Connect to RAS server and download file
Connect to network through wireless and download files
(use different protocols for modem and wireless)
With wireless card, download files size of 300MB and with modem file size of 1MB simultaneously
Confirm both wireless and modem works fine after all Powe Management operation.
1. Standby (Fn+F3)
2. Suspend (Fn+F4)
3. Hibernation (Fn+F12)
Standalone
Integration Test
File Copy
Copy files using wireless adapter
Suspend/Resume
Copy files of > 300 MB using WL adapter
Send 5 pages(graphic) of fax using modem
If Bluetooth available, transfer files to Bluetooth devices
Docking Station / Dock
Warm dock/undock
Warm UnDock/Wam dock
Warm undock the system and check for WL functions
Warm dock the system and confirm the wireless.
Standalone / Docking Station
Hot dock/undock
Hot undock the system and check for wireless functions.
Hotdock the system and confirm the wireless.
Standalone / Docking Station
Audio/Video via Wireless*1
Connect to wireless network
Play MPEG files from Network
Confirm that the file can played without much distortion and no system degaradation*1
Play MP3 files from network and confirm that the file can be played without any problem
Standalone
Wireless and Ethernet Integration
- Configure Ethernet and Wireless Adapter
- Transfer files using ftp
- Copy files using shared network drive
- Suspend the system
- Resume after one minute
- Check for both wireless and Ethernet functionality
- Hibernate the system
- Resume after one minute
- Check for both wireless and Ethernet functionality
- Transfer files using ftp
- Copy files using shared network drive
- Play MP3 files using Media Player while downloading files
Standalone / Docking Station
Wireless & WWAN Integration
- Configure WWAN and WLAN
- Enable Radio ON for WLAN
- Check for its functionality by ping to default gateway
- Power ON WWAN Device
- Check for WWAN functionality by connecting to WAN profile
- Transfer files using ftp
- Copy files using shared network drive
- Discover Bluetooth device and transfer files simultaneously.
-Suspend the system
-Resume after one minute
-Check for both wireless and WWAN functionality
-Hibernate the system
-Resume after one minute
-Check for both wireless and WWAN functionality
Standalone / Docking Station

*1 : Audio/video quality may depend on network traffic.

Repeatability [UR-2046]

Test Item Config
Result
Win 8
Win-7
Win-8
Power Management (From Radio OFF)

1. Turn Radio Off (Fn+F5)
2. Suspend (Fn+F4) the system
3. Confirm if the Radio can be Turn On (Fn+F5) (Do this for 10 trials)
4. Turn Radio Off (Fn+F5)
5. Hibernate(Fn+F12) the system
6. Confirm if the Radio can be Turn On (Fn+F5) (Do this for 10 trials)
Standalone
Power Management (From Radio ON)

1. Turn Radio On (Fn+F5)
2. Suspend (Fn+F4) the system
3. Confirm that the wireless works fine after every suspend
4. Confirm if the Radio can be Turn Off (Fn+F5) (Do this for 10 trials)
5. Turn Radio On (Fn+F5)
6. Hibernate(Fn+F12) the system
7. Confirm the functionality of wireless network
8. Confirm if the Radio can be Turn Off (Fn+F5) (Do this for 10 trials)
9. Start copying big files (>500MB)
10. Press Fn+F3
11. Check that the file copy is not affected (Do this for 10 trials)
Standalone
Power Management w/ Global Switch Function

1. Turn Global Switch to OFF
2. Suspend (Fn+F4) the system
3. Resume and Turn Global Switch to ON
4. Confirm that the wireless works fine after every suspend
5. Confirm if the Radio can be Turned ON then OFF (Fn+F5) (Do this for 10 trials)
6. Turn Global Switch to OFF
7. Hibernate(Fn+F12) the system
8. Resume and Turn Global Switch to ON
9. Confirm the functionality of wireless network
10. Confirm if the Radio can be Turned ON then OFF (Fn+F5) (Do this for 10 trials)
Standalone
Power Management w/ Software Switch Function - UR2717AP

1. Turn Software Switch to OFF

2. Suspend (Fn+F4) the system
3. Resume and Turn Software Switch to ON

4. Confirm that the wireless works fine after every suspend

5. Confirm if the Radio can be Turned ON then OFF (Fn+F5/F9) (Do this for 10 trials)

6. Turn Software Switch to OFF

7. Hibernate(Fn+F12) the system

8. Resume and Turn Software Switch to ON

9. Confirm the functionality of wireless network

10. Confirm if the Radio can be Turned ON then OFF (Fn+F5/F9) (Do this for 10 trials)
Standalone

Idling Test

Test Item Config
Result
Win 8
Win-7
Win-8
- Configure Wireless to connect to non-existing Access Point
- Wireless will check for AP connection
- Leave the system for minimum 60 minutes
Criteria :
Utility should not display "Unable to communicate with Device driver".
If such message occurs, check the following
1. No system malfunction
2. Wireless Utility should not display wrong information
3. Playing DVD/MPEG file should not be affected (No jerk etc..)
Standalone


Reliability Test

Test Item Config
Result
Win 8
Win-7
Win-8
Wireless Idling Test (LL-00830PA, CMVC #hlpct_23908)
  1. Configure multiple access points for testing
  2. Configure Wireless to connect to one Access Point(preferably Lucent AP-1000)
  3. Check for network connectivity
  4. Leave the system for 20 minutes.
  5. Confirm that the system does not looses the network connection
  6. Take the system to roam to another access point
  7. Confirm that the system connects to the another access point (configured already).
  8. Confirm that the system does not looses connectivity.
  9. Check this function in all channels settings.
Standalone
For Japanese OS : (LL-01012PA)- Verify the text shown in GUI are correct. Also verify the help file for the follwing
  • No garbage
  • No incomplete characters

Setup

1. Install OS

2. Connect to Wireless test network by TCP/IP(DHCP).

3. Enable 128bit WEP key
4. Follow the procedure to install reboot tool.

How to use Shutdown tool

Attachment "Shutdown.exe"can be seen at the bottom of the page.

Attachment "ipconfig_bat.bat"can be seen at the bottom of the page.


Recreation procedure:
1. Configure an Atlanta-4 Wireless model to connect to a wireless network with a DHCP server.
2. Create a test directory and copy the ipconfig_bat.bat and shutdown.exe files to the directory.
3. Make a shortcut of ipconfig_bat.bat in the Startup menu so that the bat file runs every boot.
4.. Boot the machine.

The machine will automatically:
a. Run ipconfig
b. Copy ipconfig results to data.txt file.
c. Display data.txt file on the screen
d. Reboot the system.
5. Follow the instraction to skip Windows Login screen.==>
The link "?????? 'PPA Test Case (S/W)', ??? 'Test Case\by Category', ?? 'Continous Boot Test (Basic Config)'" has been extracted

LongRun

Test Item Config
Result
Win 8
Win-7
Win-8
Longrun (Overnight test)
- Configure wireless and TCP/IP protocol
- With wireless card, download files size of >300MB for 4 hours continuously through FTP
- Copy files of >300MB size through shared network drive for more than 4 hours
-Copy files using wireless and simulatneously send/receive fax of 10 pages for 1 hour.
Standalone
Shutdown/Restart
- Shutdown and power ON manually for 10 times.
- Check that the wireless works fine after every boot
- Restart the system x 50 times
(Use utility to check and can be performed as overnight test).
Standalone
Power Management
- Suspend (Fn+F4) the system x 10 times
- Confirm that the wireless works fine after every suspend
- Hibernate(Fn+F12) the system x 10 times
- Confirm the functionality of wireless network
- Start copying big files (>500MB)
- Press Fn+F3 x 10 times.
- Check that the file copy is not affected
Standalone

1. More than 100 times longrun should be performed. After longrun, check log file (data.txt) and make sure no connection fail.

Test Item
Result
Win 8
Win-7
Win-8
100 times reboot w/ WEP key

Criteria :

Test Item
Wireless Network File download
Wireless Network File upload
Network performance when Media player is opened and audio file is being played
Network performance when multiple applications are opened (MS Office,Internet Explorer etc.)
Network performance with different Access points
Network performance with modem,etherent and wireless in Use simultaneously

For the following performance measurement, ensure that the system is fully configured for all system supported devices like Bluetooth, Wireless WLAN / WAN and other
Result (Enter the Value for each OS)
Test Item (LL-1055PA) Configuration
Windows 8
Win-7
Win-8
Remarks
File Download using FTP w/ Suspend
Configure TCP/IP
Wireless Network File download using FTP
Note down the source file size
Download file of 500MB and measure the download speed
Compare the source file and downloaded file size.
Suspend the system
Resume from Suspend after 1 minute
Perform file download using FTP

Repeat this step 15 times
Standalone

Docking station
Successful file download
Downloaded file must be exactly same as the source file size.
No data corruption after PM events
File Download using FTP w/ Hibernate
Configure TCP/IP
Wireless Network File download using FTP
Note down the source file size
Download file of 500MB and measure the download speed
Compare the source file and downloaded file size.
Hibernate the system
Resume from Hibernate after 1 minute
Perform file download using FTP

Repeat this step 15 times
Standalone

Docking station
Successful file download
Downloaded file must be exactly same as the source file size.
No data corruption after PM events
Wireless Network File upload
Upload file of 10MB and 100MB and measure the download speed
Standalone

Docking station
Around 2 to 3 Mbps
Wireless network performance when Media player is opened and audio file is being played
Download/upload file of 10MB and 100MB and measure the download speed
Perform this step for 15 times.
Standalone

Docking station
File download and system performance should not degrade
Wireless network performance when multiple applications are opened (MS Office,Internet Explorer etc.) Standalone

Docking station
File download and system performance should not degrade
Wireless network performance with different Access points
- Cisco 350/Cisco 1200
- Buffalo Air station
Standalone

Docking station
No performance degradation due to access point variation
Network performance with modem and wireless in Use simultaneously.
Download file of 10MB and 100MB and measure the download speed
while using modem to send fax
Standalone


Docking station
No performance degradation of wireless while accessing the other device like modem
Power ON Wireless Radio
Confirm that the WL radio powered ON immediately
Standalone Power ON - Less than 20 sec
Power OFF WL radio
Power ON WL radio after one minute
Check for link connectivity to Access Point
Standalone Link to Access Point should be less than 30 sec
1. Power ON WL Radio and connect to Access Point
2. Check that the signal quality is very good
3. Suspend the system
4. Resume from suspend after one minute
5. Check that the Access Point is linked immediately.
6. Hibernate the system
7. Resume from Hibernation after one minute
5. Check that the Access Point is linked immediately.
Standalone



Docking station
Link to Access Point should be less than 30 sec on resuming from suspend
Boot performance in AC Mode
Configure TCP/IP (Static IP)
Confirm the netowork is not connecting (Make sure system is out of RF range )
Reboot the system and check the performance
(90 sec is acceptable)
Boot performance in DC Mode
Configure TCP/IP (DHCP)
Confirm the netowork is not connecting (Make sure system is out of RF range )
Reboot the system and check the performance
(90 sec is acceptable)


UR-4166SG
1. Performance Comparison Test


Attachment "WLAN Performance Report.xlsx"can be seen at the bottom of the page.

Note: Use report sheet to input data for performance, reliability, and share connection

==Requirements==

>> 2 to 3 Wireless Router.

>> 3 location points:

a. Test Location A: Maximum Performance ( 0 to 5m away from Wireless Router)

b. Test Location B: Performance at moderate distance (10 to 15m away from Wireless Router)

c. Test Location C: Worst-case Performance (more than 20m away from Wireless Router)

>> Perform test with Test Unit and compare results with a Non-Lenovo System.

==FTP File Upload / Download Throughput Calculation==

1. Upload or Download 100 MB / 500MB/ 1GB File in FTP.

2. Use stopwatch and note the time for file transfer to complete.

3. Calculate throughput in Mbps (throughput = file size / transfer time)

4. To check signal strength, run "netsh wlan show interface" command.

6. The higher value of throughput and signal strenght means better performance.

==Online Video 1080p Streaming==

1. Pick a 15 - 30 mins 1080p video in youtube, use the same video throughout the test.

2. Play the video, and note down actual time when it completes.

3. Calculate and note down time difference (time difference = actual video complete - video duration).

4. The Lower value of time difference means better performance.

2. Reliability Comparison Test


Attachment "NetworkReliability.zip"can be seen at the bottom of the page.

>> Perform realiability test using Network ATS Script "NetworkReliability.ps1" on both Lenovo Test Unit & Non-Lenovo Test System.

>> Collect data of average Mbps for FTP file copy / Network Server File Copy / HTTP Copy.

>> Note down the results in the attached file, and compare the results.

3. Share Connection Test

==Requirement==

>> Latest build of Lenovo Settings Mobile Hotspot, WLAN / Ethernet driver.

>> 5 - 10 devices (any brand of Smart Phone/Tablet/Laptop) as client.

==Procedure==

1. Launch Lenovo Settings Mobile Hotspot and Start Sharing Connection.

2. Connect 5 to 10 client device to Test System AP (Host).

3. In the client system, run the following at the same time for 60 mins:

a. continuous ping command ex. "ping google.com -t".

b. play online video from youtube.

c. download test file from web. >> http://www.thinkbroadband.com/download.html

4. Input the data in the attached file and compare results.

4. Switching Profile

Scenario 1 (Ethernet and Wi-Fi)

>> Plug in Ethernet cable and ensure that Ethernet Profile is shown in the Networks Charms Bar

>> Connect to one Wi-Fi profile and ensure that “Connect automatically” is checked

>> Unplug the Ethernet Profile, ensure that Wi-Fi profile is in connection

>> Plug in Ethernet cable and ensure both Profiles (Ethernet and Wi-Fi) are connected

>> Disconnect Wi-Fi Profile and ensure Ethernet Profile is in connection

>> Repeat the steps for 20 times

Scenario 2 (2 Wi-Fi profiles)

>> Connect to 2 Wi-Fi profiles and ensure that “Connect automatically” is checked

>> Switch off Profile 1 router and ensure that Profile 2 is connected and having connection

>> Switch on Profile 1 router and ensure that Profile 1 is found in the Networks Charms Bar

>> Switch off Profile 2 router and ensure that Profile 1 is connected and having connection

>> Switch on Profile 2 router and ensure that Profile 2 is found in the Networks Charms Bar

>> Disconnect the current profile, Profile 1 and ensure that Profile 2 is connected automatically

>> Disconnect the current profile, Profile 2 and ensure that Profile 1 is connected automatically

>> Repeat the steps for 20 times

5. Wireless Integration / Coexistence

Scenario 1 (BIOS)

>> Connect to Wi-Fi and switch on Bluetooth

>> Boot to the BIOS menu, go to the security tab and under I/O Port Access “Disable” the Bluetooth

>> Save the settings and boot to Desktop, ensure that Wi-Fi is still in connection

>> Boot to the BIOS menu, and “Enable” the Bluetooth

>> Save the settings and boot to Desktop, ensure that Wi-Fi is still in connection

>> Repeat the steps for 20 times

Scenario 2 (Device Manager)

>> Connect to Wi-Fi and switch on Bluetooth

>> Disable the Bluetooth Device from the Device Manager and ensure Wi-Fi is still in connection

>> Enable the Bluetooth Device from the Device Manger and ensure Wi-Fi is still in connection

>> Repeat the steps for 20 times

Scenario 3 (Bluetooth and Wi-Fi)

>> Connect to Wi-Fi and switch on Bluetooth from PC settings>Network>Airplane mode

>> Switch off Bluetooth and ensure that Wi-Fi is still in connection

>> Switch on the Bluetooth and ensure that Wi-Fi is still in connection

>> Repeat the steps for 20 times

OTHER DETAILS
EUT Quantity : minimum of Two 1. "Please check Error and Warning message in Event Viewer before you start the test case and after you complete the test case, and report it to your team lead."
2. "Perform All Test in Battery Mode (Some of the test items need to perform in AC Mode, follow each Test Case)"
(All Test Activity should be performed with AMT Enabled)
Test Item Config
Result
Win 8
Win-7
Win-8
Modem/Wireless compatibiltiy Standalone
Integration Test Standalone

TEMS Reviewer Log:Yasushi Yanagisawa on 2001/07/11
Review Logs
Keywords:   None
Requirements :   None
Attached files :
NetworkReliability.zip - NetworkReliability.zip (133339 bytes, ) 07/03/2015
WLAN Performance Report.xlsx - WLAN Performance Report.xlsx (34108 bytes, ) 07/03/2015
ipconfig_bat.bat - ipconfig_bat.bat (53 bytes, ) 07/03/2015
Shutdown.exe - Shutdown.exe (133120 bytes, ) 07/03/2015

Update History:
This document is not yet modified.