Test Case

Show / Hide direct link   <MAIN> 5708PA: -draft- Wireless WAN Function test Verification - PP-81296



Version 1
Created on 13/04/2016 15:48:53  by msesha_nw
Last modified on  14/06/2016 18:49:13  by m_komatsu_nw
Summary
Preconditions
Status:   New
Priority:   High
BASIC INFORMATION
Section:S/W Test Require attachment:No
Team:Network Group:Basic Function
Lessons Learned: Phase:SIT
Locations:Inhouse: AWS and Cambridge Owned By:Eiji Ogata/LENOVO
Objective:To verify connectivity and all functionalities of Wireless WAN
CRITERIA
EUT QUANTITY
EQUIPMENT

PA Networking Facility, Activated SIM, WWAN module

Critical Attribute:WWWAN - Wireless WAN
CONFIGURATION

All supported WWAN Modules

REFERENCE PROCEDURE
STATISTIC INFO
Operating System Win10-64bit Total
Test Points
Duration
Workload
# Step actions Win10-64bit
1
OOBE Scenario
A
2

 WWAN Driver Installation Procedure

A
3

Basic Verification Test

A
4

 WWAN Firmware Update

A
5

 WWAN connection using Windows Native UI

A
6

 WWAN Auto-Firmware Switching Verification

A
7

 Cellular radio verification

A
8

 Airplane Mode Verification

A
9

 PIN Enabled and PUK Condition

A
10

 SIM PIN with PM Oerations

A
11

 Power management test with Connectivity

A
12

 Throughput Test

A
13

 VPN Connectivity

A
14

 Network Handover Test

A
DETAILS
ATTENTION!! Please follow below information.
1. Please check Error and Warning message in Event Viewer before you start the test case and after you complete the test case, and report it to your team lead, then arrange to open defect, if needed.

2. Perform All Test in Battery Mode (Some of the test items need to perform in AC Mode, follow each Test Case).

3. All Test Activity should be performed with AMT Enabled as Default (applicable on vPro enabled systems).

4. Please check Usability / Cosmetic issues (correct sentence/appropriate display/etc.) and open defect, if needed.

OOBE Scenario

Procedure

Expected

Result

1. Upon OOBE, Select time zone and click Accept for the license terms.

2. Select “Use Express Settings”

3. Enter PC name under Personalize window.

4. Click Next.

5. Join available wireless profile under Wireless window.

6. Wireless profile will ask for Security key if needed. Enter incorrect Security key.

6.1 Do this also in WWAN with available carrier network

7. Click Connect.

8. "The network password didn't work. Please check it and try again" will show.

9. Enter correct Security key, then click Connect.

10. If wireless profile is not broadcasted, click Connect to a hidden wireless network.

11. Enter information for the hidden wireless network by filling up Network name and choosing the correct Security and Encryption type.

12. Click OK.

13. Click the hidden wireless profile created, click Connect.

14. Wireless profile will ask for Security key if needed. Enter incorrect Security key.

15. Click Connect.

16. "The network password didn't work. Please check it and try again" will show.

17. Enter correct Security key, then click Connect.

18. Connection is successful if Settings window will show.

19. Click Use express settings

20. Enter your email address as Microsoft account to sign in to Windows 10.

21. Click Next.

22. Checking for a Microsoft account will show, and once the entered email was verified, enter Password.

23. Click Next.

24. Enter security verification.

25. Click Next.

26. Creating account in progress and setup finalization.

27. During First Customer Boot, try connecting WWAN.

Should be no error messages during OOBE

Should be no system hang up, bluescreen and other unnecessary error messages after connecting to available networks.

Confirm the successful sign up and connectivity.

Confirm successful connectivity in WWAN after performing OOBE

WWAN Driver Installation Procedure

Procedure

Expected

Result / Remarks

Installation (Admin Mode)

Note:

1. Before driver installation, update the firmware of the card by following the procedure mention in the firmware release notice.

1. Install WWAN Driver as per the build letter instructions.

2. Do this for all supported WWAN adapters.

- Check that the installation is successful and no error message during the installation

- Check for the correct driver version and firmware version as mentioned on the build letter

- Check the device manager for the WWAN adapter entries. Refer the device names in WWAN Build letter.

Note: WWAN device should be available under Network Adapter, Sensors and USB controllers. Any additional ports available denotes the WWAN module is not set to correct USB composition.

Device Confirmation

1. Check the Device Manager and open the Properties window for each WWAN entry.

2. 2. Open WWAN Properties in Device Manager under 'Modems'.

3. Check Driver Version is same as mentioned in build letter.

4. Go to 'Diagnostics' tab and click Query Modem button.

5. Check that firmware version is same as mentioned in build letter.

6. Check Add/Remove Program if Product Version is same as mentioned in build letter.

7. Do this for all supported WWAN adapters.

Note:

Error is prompted when "Query Modem" in Diagnostics tab is clicked a number of times. It appears that the device manager is having trouble detecting the AT port after the 1st query or if there's a sudden device removable.

Further investigation shows that when the ATI commands performed when clicking Query Modem is issued through the HyperTerminal, the modem responds successfully. Thus, only the Microsoft Device Manager exhibits the problem.

And based on Sierra's investigation, they had no issue connecting to the AT port using any terminal programs. The issue only happens with Microsoft Device Manager.

Hence Sierra concluded this is not a Sierra issue, but rather a Microsoft issue. (As Designed)

- Make sure that no Yellow Marks appear for the WWAN device.

- Make sure that all information shown for the WWAN device is correct.

- Make sure each property tab appears correctly.

Overwrite/Update Installation

1. Perform overwrite installation for the same driver version.

2. Refer to the build letter for any limitations/restrictions for overwrite installation.

3. Perform update installation from previous version to latest version.

4. Refer to the build letter for any limitations/restrictions for update installation.

5. Do this for all supported WWAN adapters.

Install WWAN Driver as per the build letter instructions

- Check that the installation is successful

- Check for the correct driver version and firmware version

Uninstallation (Admin Mode)

1. Uninstall WWAN Driver as per the build letter instructions.

2. Do this for all supported WWAN adapters.

- Check that the uninstallation is successful and no error message appears during uninstallation

Installation and Uninstallation (User Mode)

1. Install the driver in User Mode as per the build letter instructions.

2. Uninstall the driver in User Mode as per the build letter instructions.

3. Do this for all supported WWAN adapters.

- User should not be able to install the driver. Necessary message to be shown.

- User should not be able to uninstall the driver. Necessary message to be shown.

Silent installation:

1. Refer to the build for correct silent installation command and install the WWAN driver.

- Check that the installation is successful and no error message during the installation

- Check for the correct driver version and firmware version as mentioned on the build letter

Basic Verification Test

Procedure

Expected

Result

WWAN Device enable/disable function in BIOS

1. Enter to BIOS setup by pressing F1 button.

2. Security -> I/O Port Access -> Wireless WAN

3. This will have two options [Enabled, Disabled].

4. Select Enabled option

5. Confirm the functionality of the WWAN device

1. Enter to BIOS setup by pressing F1 button

2. Security -> I/O Port Access -> Wireless WAN

3. This will have two options [Enabled, Disabled].

4. Select Disabled option

5. Confirm that WWAN device cannot be used

Enabled:

- WWAN device should be available on Device Manager, OS

- WWAN device can be configured and connected to Network

Disabled:

- WWAN device should not be shown on Device Manager, OS

WWAN Device enable/disable function in Device Manager

1. Install WWAN adapter as per the build letter.

2. Disable WWAN adapter in the Device Manager.

3. Make sure that WWAN device is shown as disabled does not disappear in Device Manager. WWAN device cannot connect to Network.

4. Enable WWAN adapter in Device Manager.

5. Make sure that WWAN device is shown as enabled in Device Manager and can be connected to Network.

Enabled:

- WWAN device should be available and shown as enabled in Device Manager

- WWAN device can be connected to Network

Disabled:

- WWAN device is shown as disabled and should not disappear from Device Manager

- WWAN device cannot be connected to Network

WWAN Firmware Update

Procedure

Expected

Result

1. Extract the firmware package for the WWAN adapter.

2. Follow the instructions from the build letter or read me document and update the firmware.

Firmware update should be successful without any errors.

Note: Intel Sierra and Sierra Qualcomm adapters have multiple firmware to support various Carrier requirements

WWAN Connection using Windows

Procedure

Expected

Result

1. Insert the SIM card to the test system.

2. Power on the system.

3. Launch Start à Settings à Network & Internet à Cellular

4. Under Cellular the SIM carrier should be listed.

5. Click on the carrier and click connect

Connection should be established successfully.

WWAN Auto-Firmware Switching Verification

Procedure

Expected

Result

1. Follow the instructions from the build letter or read me document and install the driver.

2. Shutdown the system and insert the SIM [eg; AT&T].

3. Power on the system.

4. Shutdown the system and replace the SIM with a different one [eg; Verizon].

5. Power on the system.

6. Perform this for 5 different carriers.

Driver should be installed successfully without any errors.

After few secs of Power-on, firmware update should start. A progress bar at the right hand bottom corner of the screen appears. Wait until “Firmware Update Successful” message appears. Confirm the firmware version with the build letter.

After few secs of Power-on, firmware update should start. A progress bar at the right hand bottom corner of the screen appears. Wait until “Firmware Update Successful” message appears. Confirm the firmware version with the build letter.

Note:

Intel Sierra and Sierra Qualcomm adapters have multiple firmware to support various Carrier requirements.

AT&T, Verizon, Vodafone, Telstra, orange have different firmware. All other carriers use Generic firmware.

For testing this you can use non-active SIM to confirm correct firmware is updated corresponding to the carrier

Cellular Radio Verification

Procedure

Expected

Result

1. Launch Start à Settings à Network & Internet à Cellular.

2. Click on the Carrier and click connect.

3. Turn off the Cellular radio in Start à Settings à Network & Internet à Airplane Mode.

4. After 60seconds, Turn on the Mobile Broadband radio.

5. Perform steps 2 to 4 for 10 trials

Only Cellular should be turned off.

The carrier should connect successfully automatically if connect automatically was check during previous connection.

No hang-up should be encountered.

Airplane Mode Verification

Procedure

Expected

Result

1. Launch Start à Settings à Network & Internet à Cellular.

2. Click on the Carrier and click connect.

3. Turn off the Cellular radio in Start à Settings à Network & Internet à Airplane Mode.

4. After 60seconds, Turn on the Mobile Broadband radio.

5. Perform steps 1 to 4 for 10 trials

Wi-Fi and Mobile broadband should be turned off.

The carrier should connect successfully automatically if connect automatically was check during previous connection.

No hang-up should be encountered.

PIN Enabled and PUK Condition

Procedure

Expected

Result \ Remarks

PIN Enabled/ PUK Blocked Condition

1. Insert security PIN enabled SIM in the system.

2. Launch Charms à Settings à Network, Click on the Unnamed and click connect.

3. Enter correct PIN code when prompted,

4. Enter the correct PIN and click OK.

5. Shutdown and Power on the system.

6. Launch Charms à Settings à Network, Click on the Unnamed and click connect.

7. Enter incorrect PIN code for 3 trials when prompted,

8. Enter correct PUK code.

· No error should appear while enter PUK / PIN

Network should be connected successfully

The SIM card should be blocked and request for PUK code.

The SIM card should be unlocked and connected to the network successfully.

SIM PIN with PM Operations

Procedure

Expected

Result \ Remarks

1. Insert working / activated SIM in the system.

2. Launch Start à Settings à Network and Internet à Cellular à Advanced Options à SIM PIN.

3. Enter correct PIN to enable PIN

4. Suspend (S3) the system for 5 mins.

5. Resume from S3, SIM will be locked with PIN.

6. Repeat step 1 to 6.

7. Hibernate (S4) the system for 5 mins.

8. Resume from S4, SIM will be locked with PIN.

9. Enter correct PIN code.

10. Restart the system.

11. WAN profile should connect without entering PIN (Huawei, Ericsson, Sierra Qualcomm only).

Reference:For Intel-Sierra WWAN

ECR213951 W8.1:After restart need input PIN code with EM7345 WWAN

SIM PIN Behavior:

Notebooks and Ultrabooks (Windows 7 and Windows 8.1):

I.After resuming from Suspend (S3), SIM will be locked with SIM PIN.

II.After resuming from Hibernate (S4), SIM will be locked with SIM PIN.

III.After restart SIM will not be locked and can connect without entering SIM PIN. Since the module is not reset during restart.For restart PM, the SIM PIN will be prompted only for Intel WWAN. For Huawei, Sierra Qualcomm and Ericsson SIM PIN will not be prompted.

Tablets (Windows 8,1):

I.After restarting the tablet SIM will be locked with SIM PIN.

II.After resuming from S0i3 (Connected standby) SIM will not be locked and can connect without entering SIM PIN. Since the module is not reset during connected standby state.

Intel Atom processors (Bay trail) support Connected standby state and does not support Hibernate (S4). To check if the system supports connects Connected standby run the following in command prompt with Administrator privileges.

Powercfg /a

· No error should appear while enter PUK / PIN

Network should be connected successfully

Power Management

Procedure

Expected

Result

1. Launch Start à Settings à Network and Internet à Cellular.

2. Connect to the service provider.

· Connection should be established

· Connection should not be disconnected without pressing the disconnect button/or no signal is available

Note: The same test can also be performed using ATS script.

Script Name: IPConfigVerifyAfterPM (S3, S4, R)

For Tablets use VerifyAutoConnectivityForTablets script.

3. Suspend the system by Start à Power à Sleep

· System should enter suspend mode.

4. Wakeup the system after 1 minute

· System should resume/wake-up

5. Connect to the service provider

· Connection should be established

· Connection should not be disconnected without pressing the disconnect button/or signal is available

6. Repeat steps 3-5 for 20 times

· No error message should occur

·

7. Suspend system by LID close

· System should enter suspend mode

8. LID open the system after 1 minute

· System should resume

9. Connect to the service

· Connection should be established

· Connection should not be disconnected without pressing the disconnect button/or signal is available

10. Repeat steps 7-9 for 20 times

· No error message should occur

11. Hibernate the system

· System should enter hibernation mode

12. Wakeup the system after 1 minute

· System should resume/wake-up

13. Connect to the service provider

· Connection should be established

· Connection should not be disconnected without pressing the disconnect button/or signal is available

14. Repeat steps 11-13 for 20 times

· No error message should occur

15. Restart the system

· System should restart

16. Boot to OS and connect to service provider

· Connection should be established

· Connection should not be disconnected without pressing the disconnect button/or signal is available

17. Repeat steps 15-16 for 20 times

· No application crash

· No error message should occur

18. Shutdown the system

· System should shutdown properly without any error

19. Turn on the system after 1 minute

· System should boot without any error

20. Connect to the service provider

· Connection should be established

· Connection should not be disconnected without pressing the disconnect button/or signal is available

21. Repeat steps 18-20 for 20 times

· No application crash

· No error message should occur

Throughput test

Procedure

Expected

Result

1. Launch Charms à Settings àNetworks.

2. Connect to the service provider.

3. With connection established, leave the system idle for few hours (5 -10 hours) overnight.

4. After long runs perform Suspend/Hibernate/Restart.

5. Resume the system and connect again to WAN profile.

6. Download data from web (around 50 - 100 MB).

7. Check the speed and throughput of the WAN connection at www.speedtest.net

8. To check the speed and throughput click on the location arrow at www.speedtest.net

· Connection should not drop during long hours connection

· WAN profile should be connected successfully after PM operations after long runs

· Confirm that data can be downloaded successfully

· The connection speed and throughput should be under acceptable limits depending on the Service Type

Note: For Full long run and performance test, please refer to 5402PA: Network : Ethernet / WLAN / WWAN Performance measurement test - 5402PA

VPN Connectivity

Procedure

Expected

Result

1. Prepare a system configured with WWAN adapter and activated SIM.

2. Launch Charms à Settings àNetworks.

3. Connect to the service provider.

4. Confirm that the profile connects successfully.

5. Using any VPN client (Cisco, etc.), connect VPN.

6. Confirm that the user is able to connect to VPN via WWAN profile.

· Make sure that WWAN works correctly.

· Make sure that no Blue screen, Hang up and Error Messages occurred.

Note: VPN Connection on Network Flyout will take a long time to disconnect after disconnecting from the AP

ECR344907: WL2 W10: VPN takes long time to auto-disconnect when system disconnect from network

This is approved as a Help Center info(LMT-15-0651).

Network Handover Test

Procedure

Expected

Result

1. Prepare a system configured with WWAN adapter and activated SIM.

2. Launch Charms à Settings àNetworks.

3. Connect to the service provider.

4. Confirm that the profile connects successfully.

5. Drive for up to 10 miles/ Km covering multiple locations.

· Make sure that WWAN works correctly.

· Make sure that no Blue screen, Hang up and Error Messages occurred.

During the drive test confirm the connection from LTE to 3G and vice versa. No drop or failure in handover should occur.

SWPA LL-43 3G connection to LTE connection hand over fail.

ECR302223 FT:MQ2:Verizon disconnects and can't reconnect to network when switching LTE B4 and LTE B12 during handover test FR: 30%

ECR302224 FT:MQ2:AT&T disconnects and "No Serivce" appears in charms for 30 to 40 seconds when switching between UMTS and LTE bands during handover test FR: 30%

OTHER DETAILS

Please Check Event Viewer for all tests.

Perform the test in both AC and DC mode, User mode and Administrator mode.

TEMS Reviewer Log:
Review Logs
Keywords:   None
Requirements :   None
Attached files :

Update History:
Modified last: 2016-06-14 18:49:13.552, by Miyuki Komatsu_nw