Test Case

Show / Hide direct link   <MAIN> 5712PA: -draft- Mobile Hotspot Stress Test - PP-81301



Version 1
Created on 15/04/2016 16:13:48  by msesha_nw
Last modified on  14/06/2016 18:39:56  by m_komatsu_nw
Summary

 To verify Host and client connection test in usability and stress test.

Preconditions
Status:   New
Priority:   Medium
BASIC INFORMATION
Section:S/W Test Require attachment:No
Team:Network Group:Usability
Lessons Learned: Phase:SIT
Locations:Inhouse: AWS and Cambridge Owned By:Eiji Ogata/LENOVO
Objective: To verify Host and client connection test in usability and stress test.
CRITERIA

Tests to be performed in User and Admin mode.

Tests to be performed using AC and Battery.

Tests to be performed using all supported WLAN adapters.

EUT QUANTITY
EQUIPMENT

PA Network Facilities

Critical Attribute:Hotspot Feature
CONFIGURATION
REFERENCE PROCEDURE

Known Limitations/Restrictions/Information

Known Limitations
Restriction
[UR-03496SG]
Internet unable to share using Mobile Hotspot after changing WLAN adapter
Reference:ECR139996 W8:Unable to share internet using LS-MH19 aftr changing WLAN adapter
Reason:Limitation. If existing Wireless LAN card is swapped on the system and Driver is installed, User needs to restart the system before
using Lenovo Settings Mobile Hotspot.
Information
1. Desktop application support was already dropped from build GEMH08US onwards. (Ref: ECR130288-W8: The"Lenovo Mobile Hotspot"icon not show on Desktop windows)

2. Scroll bar not appears by default in Connected devices window in Mobile Hotspot
Reference:

ECR129683:Win8:Scroll bar not appear over connected devices list by default

Reason:

This is the behavior of Windows 8 List control and it is not expected to show scroll bar on metro UI when user does interact.

3. Only Ethernet (LAN) can used for sharing MH from Skylake onwards. Sharing Hotspot from Mobile Broadband is supported from OS native settings.

4. Systemwill sleepusing manual sleep and manual hibernation even'Prevent to Sleep'option is checked.

Reference:

ECR208394: W8.1:QCANFA335:SUT enters S3 even prevent sleep option check in MHSBased onMobile Hotspot - Application Spec 20140228v.2.0, page 5, bullet no.4,manual sleep and lid close operations are set by users intentionally.

Reason:

Manual sleep and hibernation of the machine is not supported in Mobile hotspot. It is by design.

5. System will not enter sleep (S3) or Hibernate (S4) in Windows 10 in AC mode, Only screen off and LED's will be ON. This is as designed by Microsoft.

Reference:

ECR369339 TH1:W10:Can't enter into suspend after Radio on Moblie Hotspot only in AC mode.

Reason:

Microsoft Bug ID: Sysdev # 6189053

this is the current expected behavior of Personal HotSpot. Screen will go off, but the system is prevented from entering sleep.

6. MH page disappears when click manage locations
Reference:

ECR364871 TH1:W10:Mobile Hotspot portion disappears after Clicking "Manage locations" under location awareness.

Reason:
Since after clicking 'manage location' , it jumps to a new page. It is working as designed
Limitation
1. W8: Mobile hotspot shows wrong IP address of Client system (FR: 80%) approved as Permanent Limitation.

Reference: ECR142326: W8: Mobile hotspot GUI shows wrong IP address of Client system (FR: 80%) (Permanent Limitation)

ECR224145 W8.1: Connected Devices in Mobile Hotspot showing different IP Address

Restriction
1. Installation in Admin mode would not reflect in User Mode and vice versa. If you would like to show it for both Users and Admin, you need to install the package manually in each User/Admin.
Ref: Lenovo Settings Build letter: "2. When you install this build manually to a system, Lenovo Settings menu is available only for a user which installs it. It works as designed. If you would like to show it for all users, you need to install it as preload module".
STATISTIC INFO
DETAILS
ATTENTION!! Please follow below information.
1. Please check Error and Warning message in Event Viewer before you start the test case and after you complete the test case, and report it to your team lead, then arrange to open defect, if needed.

2. Perform All Test in Battery Mode (Some of the test items need to perform in AC Mode, follow each Test Case).

3. All Test Activity should be performed with AMT Enabled as Default (applicable on vPro enabled systems).

4. Please check Usability / Cosmetic issues (correct sentence/appropriate display/etc.) and open defect, if needed.

Mobile Hotspot Configuration and Basic Verification

Test Procedure
Expected Result(s)
Win 10 64 Bit
Set up Mobile Hotspot
1. Connect to LAN network with internet.
2. Launch Lenovo Settings from Start GUI
3. Click Mobile Hotspoton the left pane
4.
5. Input the following to setup Hotspot.

a) [Name your Hotspot] field

b) [Create Password] field

· Confirm that Mobile Hotspot application is working properly after installation
· Confirm that no garbage, unnecessary texts (cut, not fit in text boxes, too small, etc.), buttons and links (if any) are working.
· Note by default the name of the hotspot is taken as system name.
· Without LAN or Internet or WLAN adapter installed in system ensure the following error appears, "Please make sure: 1, Your Windows account has the administrator privileges. 2, Your Internet is connected with cabled Ethernet-network. 3, Your computer has a wireless network adapter "
A
Connecting and Client connectivity
1. Press the Start sharing
2. Try connecting maximum of 8 client devices and access the hotspot:
a. Using Smart Phones, Wireless Printer, Mobile Phones w/ Wi-Fi capability, Wi-Fi Photo Frame, iPad, iPhone) with different OS (such as Android, iOS, Google Chrome, etc.) or with same OS.
b. Using other machines with different OS (such as Android, iOS, Google Chrome, etc.) or with same OS.
3. Once the devices are connected, the MAC addresses and device name should be Displayed in Mobile Hotspot's Connected Devices.
· Confirm that no BSOD or any error messages when client was connected in hotspot.
· Confirm that client can connect successfully to hotspot
· Confirm that correct client information is shown in Connected Devices.
A
Client connectivity verification
1. Client should successfully connect to the hotspot.
2. Verify that the client can connect successfully to the internet.
· Confirm that client can connect successfully to hotspot
· Confirm that client can access internet successfully
A
Mobile Hotspot Stress Test
1. Configure the Mobile Hotspot of host machine as stated on the previous test items.
2. Connect to the hotspot maximum of 8 devices simultaneously (such as Smart Phones, Wireless Printer, Mobile Phones w/ Wi-Fi capability, Wi-Fi Photo Frame, iPad, iPhone) with different OS (such as Android, iOS, Google Chrome, etc.) or with same OS)
3. Confirm connectivity success by continuous ping/browsing to the internet of each devices.
4. After confirming connectivity success of all devices, run stress test (eg. play long video using You Tube) the devices simultaneously.
5. Run the stress test minimum of 30 minutes.
6. Test the above procedures by doing the underlined scenarios below:
Check Mobile Hotspot Stress Test using Ethernet connectivity
Host Machines
Ethernet drivers
Environment
Power Options
No. of Client Devices Connected
Time
Results
Remarks
Standalone
Cradle Mode (for Tablet)
W8 32
W8 64
<mention here the test machine to be used as host machine>
<mention here the driver used in host machine>
<mention here if the host machine is in AC / DC mode>
<mention here if the host machine is in AC / DC mode>
<mention here how many minutes/hours/never put the host machine to enter sleep mode>
<mention here the number and specify all the devices connected to the host machine>
<mention here how many minutes/hours the stress test of the connected devices>
<mention here if PASS / FAIL>
<mention here if PASS / FAIL>
<mention here if any symptoms/observation was seen,
examples:
- No connection drop
- The 5th device cannot connect anymore
- After 45mins, while trying to connect 5th machine, all system disconnected with host system going to limited connectivity. If we do not play Youtube, then no such problem happens.>
<just add rows for other test machines and mention the same as above>
<just add rows for other test machines and mention the same as above>
<just add rows for other test machines and mention the same as above>
<just add rows for other test machines and mention the same as above>
<just add rows for other test machines and mention the same as above>
<just add rows for other test machines and mention the same as above>
<just add rows for other test machines and mention the same as above>
<just add rows for other test machines and mention the same as above>
<just add rows for other test machines and mention the same as above>
<just add rows for other test machines and mention the same as above>
YouTube playback on mobile Hotspot client systems
Test Procedure
Expected Result(s)
Win8 64 Bit
YouTube playback on Client systems:
1. Launch Lenovo Settings from Start GUI
2. Click Mobile Hotspot
3. Connect to LAN network with internet connectivity.
4. Input the following to setup Hotspot.
a) [Name your Hotspot] field
b) [Create Password] field
5. Press the Start sharing
6. Try connecting maximum of 5 client devices and access the hotspot:
6.1 Using Smart Phones, Wireless Printer, Mobile Phones w/ Wi-Fi capability, Wi-Fi Photo Frame, iPad, iPhone) with different OS (such as Android, iOS, Google Chrome, etc.) or with same OS.
6.2 Using other machines with different OS (such as Android, iOS, Google Chrome, etc.) or with same OS
7. Once the devices are connected, the MAC addresses and device name should be shown in Mobile Hotspot's Connected Devices.
8. Client should successfully connected in hotspot.
9. Verify that the client can connect successfully to the internet.
10.Open browser & play HD video(search for 1080P video) on You tube in Client systems.
11.Confirm video plays smooth & no loss in Connectivity.
Result:
Client-playback
240p
360p
480p
720p
1080p
1
2
3
4
5
Simultaneous playback result:
Client-playback
1
2
3
4
5
Result
240p
360p
480p
720p
1080p
Client -Playback
Video Quality
1
240p
360p
360p
480p
720p
1080p
720p
2
360p
480p
480p
480p
720p
1080p
720p
3
480p
480p
1080p
480p
720p
1080p
720p
4
720p
720p
1080p
720p
480p
720p
720p
5
1080p
720p
720p
720p
1080p
720p
1080p
Result
Note:
Repeat the test steps & update the table with HD playback on Host -Machine also.
Repeat test steps with both Ethernet & WLAN connection.
- Confirm that HD video plays smooth in all systems.
-Confirm no loss or drop in connectivity.
-Confirm stability of Host machine
-Confirm that the connected devices should not disconnect and continue its functionality (the video streaming should continue).
A
File transfer and Video playback Stress test
Test Procedure
Expected Result(s)
Win8 32 Bit
YouTube playback and file transfer on host and Client systems:
Setup:
1. Set up an Access point
2. Prepare a system with static IP and share folder location. Connect to the AP
3. Connect the test system [host] to the AP via LAN.
1. Launch Lenovo Settings from Start GUI
2. Click Mobile Hotspot
3. Connect to LAN network with internet connectivity to the AP.
4. Input the following to setup Hotspot.
a) [Name your Hotspot] field
b) [Create Password] field
5. Press the Start sharing
6. Try connecting maximum of 5 client devices and access the hotspot:
6.1 Using Smart Phones, Wireless Printer, Mobile Phones w/ Wi-Fi capability, Wi-Fi Photo Frame, iPad, iPhone) with different OS (such as Android, iOS, Google Chrome, etc.) or with same OS.
6.2 Using other machines with different OS (such as Android, iOS, Google Chrome, etc.) or with same OS
7. Once the devices are connected, the MAC addresses and device name should be shown in Mobile Hotspot's Connected Devices.
8. Client should successfully connected in hotspot.
9. Verify that the client can connect successfully to the internet.
10.Open browser & play HD video(search for 1080P video) on You tube in host and Client systems.

11. Simultaneously transfer files of large sizes 512 MB, 1 GB , 5 GB between host and share system, client and host system.

12.Confirm video plays smooth & no loss in Connectivity.
- Confirm that HD video plays smooth in all systems.
-Confirm no loss or drop in connectivity.
-Confirm stability of Host machine
-Confirm that the connected devices should not disconnect and continue its functionality (the video streaming should continue).
A
Power Management Verification

Test Procedure
Expected Result(s)
Win8 64 Bit
Basic PM Operations
Suspend / Resume
1. Perform Suspend / Resume in Mobile Hotspot system.
2. Connect any supported client to the Mobile Hotspot system through hotspot connection.
Hibernate / Resume
1. Perform Hibernate / Resume in Mobile Hotspot system.
2. Connect any supported client to the Mobile Hotspot system through hotspot connection.
Lid Close / Resume
1. Perform Lid Close / Resume in Mobile Hotspot system.
2. Connect any supported client to the Mobile Hotspot system through hotspot connection.
Reboot
1. Perform Reboot in Mobile Hotspot system.
2. After Mobile Hotspot system reboot.
3. Connect any supported client to the Mobile Hotspot system through hotspot connection.
System Shutdown
1. Perform Shutdown in Mobile Hotspot system.
2. Boot again Mobile Hotspot system.
3. Connect any supported client to the Mobile Hotspot system through hotspot connection.
Perform each PM operations for 5-10 trials
- Confirm that connection is established successfully after PM operations.
- Confirm that all functions (buttons, links, etc.) can be performed successfully after PM operations.
- Confirm that no error messages should show after PM operations.
A
Connected Standby Mode Operations (for Connected Standby supported systems only)
Note:
Make sure to perform this item by the following environment below:
- AC/DC Mode
- AC Docked in Cradle (for Tablet systems)
- AC USB (for Tablet systems)
1. Connect up to maximum of 8 clients and access the hotspot
2. Using Smart Phones, Wireless Printer, Mobile Phones w/ Wi-Fi capability, Wi-Fi Photo Frame) with different OS (such as Android, iOS, Google Chrome, etc.) or with same OS.
2.3 Using other machines with different OS (such as Android, iOS, Google Chrome, etc.) or with same OS
2.4 While the devices are connected to hotspot, play video streaming using You Tube.
3. Let the host system enter in Connected Standby mode.
4. Observe the connected devices.
Ref. ECR145387, W8:Mobile Hotspot disconnects when enter into S3 in AC/DC mode
- Confirm that the connected devices should not disconnect and continue its functionality (the video streaming should continue).
A
Event Viewer / Lenovo Hotspot Service Verification
Suspend / Resume
1. Perform Suspend in Mobile Hotspot system.
2. Resume after 1 minute from Suspend Mobile Hotspot system.
3. Go to Event Viewer:
3.1 Right click My Computer > Manage > Event Viewer
3.2 Confirm that there are NO Critical / Error / Warning messages for Lenovo Hotspot Service.
4. Go to Problem Reports:
4.1 Go to Control Panel then type Problem Reports in "Search Control Panel" textbox
4.2 Open "View all problem reports"
4.3 Confirm that there are NO crash / stopped working messages for Lenovo Hotspot Service.
5. Connect any supported client to the Mobile Hotspot system through hotspot connection.
Hibernate / Resume
1. Perform Hibernate in Mobile Hotspot system.
2. Resume after 1 minute from Hibernate Mobile Hotspot system.
3. Go to Event Viewer:
3.1 Right click My Computer > Manage > Event Viewer
3.2 Confirm that there are NO Critical / Error / Warning messages for Lenovo Hotspot Service.
4. Go to Problem Reports:
4.1 Go to Control Panel then type Problem Reports in "Search Control Panel" textbox
4.2 Open "View all problem reports"
4.3 Confirm that there are NO crash / stopped working messages for Lenovo Hotspot Service.
5. Connect any supported client to the Mobile Hotspot system through hotspot connection.
Lid Close / Resume
1. Perform Lid Close in Mobile Hotspot system.
2. Resume after 1 minute from Lid Close Mobile Hotspot system.
3. Go to Event Viewer:
3.1 Right click My Computer > Manage > Event Viewer
3.2 Confirm that there are NO Critical / Error / Warning messages for Lenovo Hotspot Service.
4. Go to Problem Reports:
4.1 Go to Control Panel then type Problem Reports in "Search Control Panel" textbox
4.2 Open "View all problem reports"
4.3 Confirm that there are NO crash / stopped working messages for Lenovo Hotspot Service.
5. Connect any supported client to the Mobile Hotspot system through hotspot connection.
Reboot
1. Perform Reboot in Mobile Hotspot system.
2. After Mobile Hotspot system reboot.
3. Go to Event Viewer:
3.1 Right click My Computer > Manage > Event Viewer
3.2 Confirm that there are NO Critical / Error / Warning messages for Lenovo Hotspot Service.
4. Go to Problem Reports:
4.1 Go to Control Panel then type Problem Reports in "Search Control Panel" textbox
4.2 Open "View all problem reports"
4.3 Confirm that there are NO crash / stopped working messages for Lenovo Hotspot Service.
5. Connect any supported client to the Mobile Hotspot system through hotspot connection.
System Shutdown
1. Perform Shutdown in Mobile Hotspot system.
2. Boot again Mobile Hotspot system.
3. Go to Event Viewer:
3.1 Right click My Computer > Manage > Event Viewer
3.2 Confirm that there are NO Critical / Error / Warning messages for Lenovo Hotspot Service.
4. Go to Problem Reports:
4.1 Go to Control Panel then type Problem Reports in "Search Control Panel" textbox
4.2 Open "View all problem reports"
4.3 Confirm that there are NO crash / stopped working messages for Lenovo Hotspot Service.
5. Connect any supported client to the Mobile Hotspot system through hotspot connection.
Perform each PM operations for 5-10 trials
Connected Standby Mode (for Connected Standby supported systems only)
1. Connect up to maximum of 8 clients and access the hotspot
2. Using Smart Phones, Wireless Printer, Mobile Phones w/ Wi-Fi capability, Wi-Fi Photo Frame) with different OS (such as Android, iOS, Google Chrome, etc.) or with same OS.
2.3 Using other machines with different OS (such as Android, iOS, Google Chrome, etc.) or with same OS
2.4 While the devices are connected to hotspot, play video streaming using You Tube.
3. Let the host system enter in Connected Standby mode.
4. Resume after 1 minute.
5. Go to Event Viewer:
5.1 Right click My Computer > Manage > Event Viewer
5.2 Confirm that there are NO Critical / Error / Warning messages for Lenovo Hotspot Service.
6. Go to Problem Reports:
6.1 Go to Control Panel then type Problem Reports in "Search Control Panel" textbox
6.2 Open "View all problem reports"
6.3 Confirm that there are NO crash / stopped working messages for Lenovo Hotspot Service.
7. Connect any supported client to the Mobile Hotspot system through hotspot connection.
Perform each PM operations for 5-10 trials
OTHER DETAILS
TEMS Reviewer Log:
Review Logs
Keywords:   None
Requirements :   None
Attached files :

Update History:
Modified last: 2016-06-14 18:39:56.705, by Miyuki Komatsu_nw