urllib: InsecurePlatformWarning

            if isinstance(e.reason, _SSLError):
                # This branch is for urllib3 v1.22 and later.
>               raise SSLError(e, request=request)
E               SSLError: HTTPSConnectionPool(host='xxx.yyy.zzz.com', port=443): Max retries exceeded with url: /api/auth/login/prepare/mobile/ (Caused by SSLError(SSLError(1, '_ssl.c:510: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed'),))

ENV/local/lib/python2.7/site-packages/requests/adapters.py:514: SSLError
=============================== warnings summary ===============================
test_suite/api_admin_backend/test_api_admin_backend_admin_user.py::TestOchaAdminBackendAdminUser::test_api_admin_backend_admin_user_query
  /var/lib/jenkins/workspace/xxxxx/ENV/local/lib/python2.7/site-packages/urllib3/util/ssl_.py:355: SNIMissingWarning: An HTTPS request has been made, but the SNI (Server Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
    SNIMissingWarning

test_suite/api_admin_backend/test_api_admin_backend_admin_user.py::TestOchaAdminBackendAdminUser::test_api_admin_backend_admin_user_query
  /var/lib/jenkins/workspace/xxxxx/ENV/local/lib/python2.7/site-packages/urllib3/util/ssl_.py:148: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
    InsecurePlatformWarning

-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file: /var/lib/jenkins/workspace/beepos-api-test-dev/output/result.xml -
============== 153 failed, 2 warnings, 306 rerun in 74.37 seconds ==============
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Finished: UNSTABLE

Solution: in your requirements.txt, change 'requests' to 'requests[security]'


How we test mobile products

I did a sharing to local teams about how we test mobile products. It has some good practices you may think it is useful (sensitive slides/information is removed from the slides before uploading). You can view it directly from below URL:

Perform long press, select all, and cut to clear text on iOS

el1 = driver.find_element_by_accessibility_id('CashierReceiptEditViewController.textViewElement')
orig_receipt_footer = el1.text
el1.click()
# el1.clear() -- https://github.com/appium/appium/issues/9752
# el1.send_keys('xxx')

actions = TouchAction(driver)
actions.long_press(el1)
actions.perform()

# sleep(1)
# print(driver.page_source)

el3 = driver.find_element_by_xpath('//XCUIElementTypeMenuItem[@name="Select All"]')
el3.click()

el4 = driver.find_element_by_xpath('//XCUIElementTypeMenuItem[@name="Cut"]')
el4.click()

iOS Appium Automation Testing Setup Tips

  • Good materials:
  • Follow https://github.com/appium/appium-xcuitest-driver/blob/master/docs/real-device-config.md to setup:
    • libimobiledevice – install using brew install libimobiledevice --HEAD
    • ios-deploy – install using npm install -g ios-deploy
    • { xcodeOrgId: <Team ID>, xcodeSigningId: iPhone Developer }
      # <Team ID> is a 10 chars ID from your team account. If you have fastlane build configure, you can find from there.
    • If you meet:
      • [XCUITest] [ 70%] VerifyingApplication [XCUITest] ‘. Stderr: ‘2019-02-25 18:56:53.483 ios-deploy[15543:11667215] [ !! ] Error 0xe8008017: A signed resource has been added, modified, or deleted. AMDeviceSecureInstallApplication(0, device, url, options, install_callback, 0) [XCUITest] ‘. [XCUITest] Error: Could not install app: ‘Command ‘ios-deploy –id e03ea2d4ee568fdb8a1a3ea7a6cd5573d3684d6e –bundle /var/folders/qw/3n_wydks4wx4pw_ntsm2h9xrzhfnbl/T/2019125-63761-15x74l9.m7xk/Payload/xxx-TH.app’ exited with code 253′

        Check if your ipa contains any special chars in the file name, mine:
        codesign -v –verbose /var/folders/qw/3n_wydks4wx4pw_ntsm2h9xrzhfnbl/T/2019125-63761-15x74l9.m7xk/Payload/xxx-TH.app /var/folders/qw/3n_wydks4wx4pw_ntsm2h9xrzhfnbl/T/2019125-63761-15x74l9.m7xk/Payload/xxx-TH.app: a sealed resource is missing or invalid file added: /private/var/folders/qw/3n_wydks4wx4pw_ntsm2h9xrzhfnbl/T/2019125-63761-15x74l9.m7xk/Payload/xxx-TH.app/Images/Group∩╝ì02.png file added: /private/var/folders/qw/3n_wydks4wx4pw_ntsm2h9xrzhfnbl/T/2019125-63761-15x74l9.m7xk/Payload/xxx-TH.app/Images/Group∩╝ì01.png file added: /private/var/folders/qw/3n_wydks4wx4pw_ntsm2h9xrzhfnbl/T/2019125-63761-15x74l9.m7xk/Payload/xxx-TH.app/Images/Group∩╝ì03.png file added: /private/var/folders/qw/3n_wydks4wx4pw_ntsm2h9xrzhfnbl/T/2019125-63761-15x74l9.m7xk/Payload/xxx-TH.app/Images/Group∩╝ì04.png file missing: /private/var/folders/qw/3n_wydks4wx4pw_ntsm2h9xrzhfnbl/T/2019125-63761-15x74l9.m7xk/Payload/xxx-TH.app/Images/Group-04.png file missing: /private/var/folders/qw/3n_wydks4wx4pw_ntsm2h9xrzhfnbl/T/2019125-63761-15x74l9.m7xk/Payload/xxx-TH.app/Images/Group-03.png file missing: /private/var/folders/qw/3n_wydks4wx4pw_ntsm2h9xrzhfnbl/T/2019125-63761-15x74l9.m7xk/Payload/BeePOS-TH.app/Images/Group-02.png file missing: /private/var/folders/qw/3n_wydks4wx4pw_ntsm2h9xrzhfnbl/T/2019125-63761-15x74l9.m7xk/Payload/xxx-TH.app/Images/Group-01.png
        After fixing the file name issue and rebuild I can install the .ipa to a real device.
    • If you meet
      • [XCUITest] ‘. Stderr: ‘2019-02-25 23:45:33.230 ios-deploy[22888:11854485] [ !! ] Error 0xe8008016: The executable was signed with invalid entitlements. AMDeviceSecureInstallApplication(0, device, url, options, install_callback, 0) [XCUITest] ‘. [XCUITest] Error: Could not install app: ‘Command ‘ios-deploy –id e03ea2d4ee568fdb8a1a3ea7a6cd5573d3684d6e –bundle /var/folders/qw/3n_wydks4wx4pw_ntsm2h9xrzhfnbl/T/2019125-17875-1b1nr5g.ab93/Payload/BeePOS-TH-Enterprise.app’ exited with code 253′ [XCUITest] at IOSDeploy.install (/Applications/Appium.app/Contents/Resources/app/node_modules/appium-xcuitest-driver/lib/ios-deploy.js:40:13) [XCUITest] at process._tickCallback (internal/process/next_tick.js:68:7) [XCUITest] Not clearing log files. Use clearSystemFiles capability to turn on.

        You have to build your .ipa with a real provisioning profile. ‘wildcard provisioning profile’ will not work here.
    • If you meet
      • “info XCUITest xcodebuild exited with code ’65’ and signal ‘null'”, it means you fail to install and run WebDriverAgent on your real device. Follow https://github.com/imurchie/appium-xcuitest-driver/blob/isaac-rs/docs/real-device-config.md#basic-manual-configuration (I followed the “Full manual configuration” section:
        •     $ which appium
              /path/where/installed/bin/appium
          
              mkdir -p Resources/WebDriverAgent.bundle
              ./Scripts/bootstrap.sh -d
        • Open WebDriverAgent.xcodeproj and build the project:
          • You will need to change the “Product Bundle Identifier” from com.facebook.WebDriverAgentRunner to something that Xcode will accept
          • Then cd /usr/local/lib/node_modules/appium/node_modules/appium-xcuitest-driver/WebDriverAgent
            • xcodebuild -project WebDriverAgent.xcodeproj -scheme WebDriverAgentRunner -destination 'id=<udid>' test
              
          • To run WebDriverAgent, just (‘without the test’):
            • xcodebuild -project WebDriverAgent.xcodeproj -scheme WebDriverAgentRunner -destination 'id=<udid>'
  • Your appium desired capabilities should be like,
    • driver = webdriver.Remote(
          command_executor=EXECUTOR,
          desired_capabilities={
              'app': IOS_APP_PATH,
              'platformName': 'iOS',
              'automationName': 'XCUITest',
              "xcodeOrgId": "yyyyyxxxxx",
              'udid': 'e03ea2d4ee568fdb8a1a3ea7a6cdyyyyyyxxxxxd',
              "xcodeSigningId": "iPhone Developer",
              "updateWDABundleId": "com.seagroup.WebDriverAgentRunner",
              'platformVersion': os.getenv('IOS_PLATFORM_VERSION') or '12.1.4',
              'deviceName': os.getenv('IOS_DEVICE_NAME') or 'iPad',
              "bootstrapPath": "/usr/local/lib/node_modules/appium/node_modules/appium-xcuitest-driver/WebDriverAgent",
              "agentPath": "/usr/local/lib/node_modules/appium/node_modules/appium-xcuitest-driver/WebDriverAgent/WebDriverAgent.xcodeproj"
          }
      )
  • x

Android Client UI automation tips

  • Defines the fixture in conftest.py, say,
    • @pytest.fixture(scope="function")
      def driver(request):
          desired_caps = {
              'platformName': 'Android',
              'platformVersion': PLATFORM_VERSION,
              'app': APP_PATH,
              'unicodeKeyboard': True,
              'resetKeyboard': True,
              'deviceName': 'T21718CJ40232',
              # 'dontStopAppOnReset': True
              # 'autoGrantPermissions': True,
          }
          driver = webdriver.Remote(APPIUM_URL, desired_caps)
          driver.implicitly_wait(5)
      
          def fin():
              driver.quit()
      
          request.addfinalizer(fin)
          return driver  # provide the fixture value
  • Define how pytest should run in pytest.ini, say, rerun failed cases,
    • addopts = -v --reruns 3 --html=report.html --self-contained-html
  • Define pytest.mark in pytest.ini,
    • markers =
        tier1: smoke testing, test core function
        tier2: sanity testing, test main function (core + high-frequency operations)
        qa: run the test case on QA server only
        dev: run the test case on Dev server only
  • If you wanted to pass env. into the python, use,
    • os.getenv('APPIUM_URL') if os.getenv('APPIUM_URL') else 'http://localhost:4723/wd/hub'
    • @pytest.mark.skipif(os.getenv('OLD_CART_UI'), reason='skip it because it is old cart UI')
  • If you want to scroll your window to find out element ‘xyz’, you can use,
    • driver.find_element_by_android_uiautomator(PATH.COMMON_SCROLL_LEFT_PANEL.format(name))
      
      
      COMMON_SCROLL_LEFT_PANEL = "new UiScrollable(new UiSelector().scrollable(true).instance(0))." \
                                 "scrollIntoView(new UiSelector()." \
                                 "textContains(\"" + '{!s}' + "\").instance(0))"
      # If you wanted to scroll the second scrollable="true" element, use instance(1)
      # If you want to know how scrollable element in the page, print it via driver.page_source and search 'scrollable="true"'
  • http://www.xpathtester.com/xpath is a nice xpath tool.
  • xpath relative example, say,
    • INPUT_PRICE = "//android.widget.RelativeLayout[" \
                    "@resource-id='com.xxx.yyy:id/oc_item_default_price']" \
                    "//android.widget.EditText[@resource-id='com.xxx.yyy:id/oc_text_row_edit']"
    • COMMON_FIND_BY_TEXT = 'new UiSelector().text("{!s}")'
      COMMON_FIND_BY_TEXT_CONTAINS = 'new UiSelector().textContains("{!s}")'
  • Find element using uiautomator,
    • COMMON_FIND_BY_TEXT = ‘new UiSelector().text(“{!s}”)’
      COMMON_FIND_BY_TEXT_CONTAINS = ‘new UiSelector().textContains(“{!s}”)’
    • find_element_by_android_uiautomator
  • Sometimes appium is unable to refresh the page_source, try to bring it background then appium will refresh
    • driver.background_app(1)
      # sometimes you may need to try to seelp(seconds) after the background command to await it completes
  • A typical android client UI automation project directory structure.
    • Screen Shot 2019-02-24 at 9.16.16 PM
      • test_suite: it defines the test cases, it is like your test spec
        actions: it contains all of the ACTIONS, like find an element, scroll and click it.
      • labels: it contains the TEXTS in XXX App, say, ‘Tax rates will be autoxxx’
      • configs: it defines the test configurations, say, which apk you wanted to test
      • paths: it defines the PATHS how can you find the elements, say, ‘com.xxx.yyy:id/oc_switch_active’
      • conftest.py: it defines python fixtures for the test run
      • pytest.ini: it configures how pytest runs
  • Hide soft keyboard:
    • ‘unicodeKeyboard’: True,
    • ‘resetKeyboard’: True,
    • or
      driver.hide_keyboard() # case by case

Setup openstf on CentOS 7

  1. Firstly follow https://linuxconfig.org/install-gnome-gui-on-rhel-7-linux-server to install GNOME GUI for CentOS 7
  2. Follow https://www.linuxtechi.com/install-configure-vnc-server-centos-7-rhel-7/ to setup VNC Server (systemctl restart vncserver@:3.service)
  3. Use VNC Client to connect
  4. https://github.com/openstf/stf Install the requirements
    1. Node.js >= 6.9 (latest stable version preferred)
    2. ADB properly set up
    3. RethinkDB >= 2.2 (rethinkdb –bind all)
    4. GraphicsMagick (for resizing screenshots)
    5. ZeroMQ libraries installed
    6. Protocol Buffers libraries installed
    7. yasm installed (for compiling embedded libjpeg-turbo)
    8. pkg-config so that Node.js can find the libraries
  5. I have to use make build install most of the above softwares
  6. When you start your adb device, you need to choose Graphics as Software https://stackoverflow.com/questions/42728353/cannot-start-android-device-emulator-on-linux/42903352
  7. Whitelist all of the ports for 7xxx.
  8. stf local –public-ip 103.78.77.65 –allow-remote

Install openstf on Mac

Follow https://github.com/openstf/stf

Install the following requirements firstly:

Note, Node.js 10.13 or 11.2 doesn’t work. I had to rollback to 8.13 to get it work

➜ Downloads npm install -g stf
npm WARN deprecated @slack/client@3.16.0: v3.x and lower are no longer supported. see migration guide: https://github.com/slackapi/node-slack-sdk/wiki/Migration-Guide-for-v4
npm WARN deprecated node-uuid@1.4.8: Use uuid module instead
npm WARN deprecated cryptiles@2.0.5: This version is no longer maintained. Please upgrade to the latest version.
npm WARN deprecated boom@2.10.1: This version is no longer maintained. Please upgrade to the latest version.
npm WARN deprecated hoek@2.16.3: This version is no longer maintained. Please upgrade to the latest version.
npm WARN deprecated ejs@0.8.8: Critical security bugs fixed in 2.5.5
/usr/local/bin/stf -> /usr/local/lib/node_modules/stf/bin/stf

> dtrace-provider@0.8.7 install /usr/local/lib/node_modules/stf/node_modules/dtrace-provider
> node-gyp rebuild || node suppress-error.js

ACTION binding_gyp_ndtp_target_build_ndtp .
TOUCH Release/obj.target/ndtp.stamp

> bufferutil@1.3.0 install /usr/local/lib/node_modules/stf/node_modules/bufferutil
> node-gyp rebuild

CXX(target) Release/obj.target/bufferutil/src/bufferutil.o
In file included from ../src/bufferutil.cc:16:
In file included from ../../nan/nan.h:190:
../../nan/nan_maybe_43_inl.h:88:15: error: no member named ‘ForceSet’ in ‘v8::Object’
return obj->ForceSet(GetCurrentContext(), key, value, attribs);
~~~ ^
In file included from ../src/bufferutil.cc:16:
In file included from ../../nan/nan.h:196:
In file included from ../../nan/nan_new.h:189:
../../nan/nan_implementation_12_inl.h:345:28: warning: ‘New’ is deprecated [-Wdeprecated-declarations]
return v8::StringObject::New(value).As<v8::StringObject>();

On Mac OS, you can use homebrew to install most of the dependencies:

brew install rethinkdb graphicsmagick zeromq protobuf yasm pkg-config

Actually I installed rethinkdb by myself and graphicsmagick zeromq protobuf yasm pkg-config with brew install.

Then,

rethinkdb

stf local

 

Use dnsmasq to setup DNS server for TCP testing

In https://luohuahuang.org/2018/08/23/mitmproxy/ I introduced a tool to setup HTTP(s) proxy to test our new server. Our Android client is using HTTPS protocol, however our iOS client is using TCP protocol. mitmproxy is for HTTP(s) protocol only.

Found one tool https://gist.github.com/ogrrd/5831371 for this purpose.

Follow steps on https://gist.github.com/ogrrd/5831371 and http://www.natemcintyre.com/blog/2016/8/25/configuring-your-own-dns-server-on-mac-os-x to setup

  1. brew install dnsmasq
  2. mkdir -pv $(brew –prefix)/etc/
  3. echo ‘address=/.dev/127.0.0.1’ > $(brew –prefix)/etc/dnsmasq.conf
  4. sudo cp -v $(brew –prefix dnsmasq)/homebrew.mxcl.dnsmasq.plist /Library/LaunchDaemons
  5. sudo launchctl load -w /Library/LaunchDaemons/homebrew.mxcl.dnsmasq.plist
  6. sudo mkdir -v /etc/resolver
  7. sudo bash -c ‘echo “nameserver 127.0.0.1” > /etc/resolver/dev’
  8. Configure hostname to IP address mapping via the local hosts file.
    1. Command: vim /etc/hosts
    2. This step can vary depending on how you have ‘dnsmasq’ configured, but with default configurations the software will look to the servers /etc/hosts file to serve as the hostname/ip registrar.
  9. System Preferences > Network > Wi-Fi (or whatever you use) > Advanced… > DNS > add 127.0.0.1 to top of the list.
  10. Image Pasted at 2018-8-23 17-05.png