1
0
Fork 0
fenix/automation/taskcluster/decision_task.py

156 lines
4.9 KiB
Python
Raw Normal View History

# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
"""
Decision task for nightly releases.
"""
from __future__ import print_function
import argparse
import datetime
import os
import re
import taskcluster
Configure either geckoview beta or nightly at compile-time (#4851) * Remove "abi" product flavor and introduce "engine" product flavor. This patch will allow us to build Fenix against GeckoView Nightly and GeckoView Beta by introducing a new flavor dimension: engine = [geckoNightly, geckoBeta]. In addition to that it adds a "fenix" prefix to the nightly, beta and production flavors to reduce the ambiguity between fenix beta/nightly and GeckoView beta/nightly. For now the build types have the following engine variants enabled: **debug**: geckoNightly, geckoBeta Both variants enabled for local development and testing. **forPerformanceTest**: geckoNightly, geckoBeta Both variants enabled unless the perf team only cares about Nightly (tbd) **fenixNightlyLegacy**: geckoBeta Uses GeckoView Beta for now - the same version we ship production builds with (same behavior as before). This release type will eventualyl be decommissioned once we switch to a separate Nightly app on Google Play. **fenixNightly**: geckoBeta Uses GeckoView Beta for now - the same version we ship production builds with (same behavior as before). Changing this build to use GeckoView Nightly is currently being discussed. **fenixBeta**: geckoBeta Fenix Beta uses GeckoView Beta. **fenixProduction** Fenix Production uses GeckoView Beta (69) currently. * gradle.py/variant.py: Replace "abi" with "engine". * Disable enableUnitTestBinaryResources until we can switch to Android Gradle plugin 3.5. * Fenix nightly should use both geckoview nightly and beta * Updates automation to use apk splitting and support different engine
2019-08-21 17:32:01 +02:00
from lib.gradle import get_variant
2019-04-23 14:21:21 +02:00
from lib.tasks import (
fetch_mozharness_task_id,
schedule_task_graph,
TaskBuilder,
)
from lib.chain_of_trust import (
populate_chain_of_trust_task_graph,
populate_chain_of_trust_required_but_unused_files
)
def pr(builder):
tasks = []
2019-03-27 15:23:03 +01:00
Configure either geckoview beta or nightly at compile-time (#4851) * Remove "abi" product flavor and introduce "engine" product flavor. This patch will allow us to build Fenix against GeckoView Nightly and GeckoView Beta by introducing a new flavor dimension: engine = [geckoNightly, geckoBeta]. In addition to that it adds a "fenix" prefix to the nightly, beta and production flavors to reduce the ambiguity between fenix beta/nightly and GeckoView beta/nightly. For now the build types have the following engine variants enabled: **debug**: geckoNightly, geckoBeta Both variants enabled for local development and testing. **forPerformanceTest**: geckoNightly, geckoBeta Both variants enabled unless the perf team only cares about Nightly (tbd) **fenixNightlyLegacy**: geckoBeta Uses GeckoView Beta for now - the same version we ship production builds with (same behavior as before). This release type will eventualyl be decommissioned once we switch to a separate Nightly app on Google Play. **fenixNightly**: geckoBeta Uses GeckoView Beta for now - the same version we ship production builds with (same behavior as before). Changing this build to use GeckoView Nightly is currently being discussed. **fenixBeta**: geckoBeta Fenix Beta uses GeckoView Beta. **fenixProduction** Fenix Production uses GeckoView Beta (69) currently. * gradle.py/variant.py: Replace "abi" with "engine". * Disable enableUnitTestBinaryResources until we can switch to Android Gradle plugin 3.5. * Fenix nightly should use both geckoview nightly and beta * Updates automation to use apk splitting and support different engine
2019-08-21 17:32:01 +02:00
variant = get_variant('debug', 'geckoNightly')
tasks.append(builder.craft_assemble_pr_task(variant))
tasks.append(builder.craft_test_pr_task(variant))
2019-03-27 15:23:03 +01:00
for craft_function in (
builder.craft_detekt_task,
builder.craft_ktlint_task,
builder.craft_lint_task,
builder.craft_compare_locales_task,
2019-03-27 15:23:03 +01:00
):
tasks.append(craft_function())
2019-03-27 15:23:03 +01:00
return tasks
def push(builder):
2019-09-13 08:19:42 +02:00
all_tasks = pr(builder)
all_tasks.append(builder.craft_ui_tests_task())
Configure either geckoview beta or nightly at compile-time (#4851) * Remove "abi" product flavor and introduce "engine" product flavor. This patch will allow us to build Fenix against GeckoView Nightly and GeckoView Beta by introducing a new flavor dimension: engine = [geckoNightly, geckoBeta]. In addition to that it adds a "fenix" prefix to the nightly, beta and production flavors to reduce the ambiguity between fenix beta/nightly and GeckoView beta/nightly. For now the build types have the following engine variants enabled: **debug**: geckoNightly, geckoBeta Both variants enabled for local development and testing. **forPerformanceTest**: geckoNightly, geckoBeta Both variants enabled unless the perf team only cares about Nightly (tbd) **fenixNightlyLegacy**: geckoBeta Uses GeckoView Beta for now - the same version we ship production builds with (same behavior as before). This release type will eventualyl be decommissioned once we switch to a separate Nightly app on Google Play. **fenixNightly**: geckoBeta Uses GeckoView Beta for now - the same version we ship production builds with (same behavior as before). Changing this build to use GeckoView Nightly is currently being discussed. **fenixBeta**: geckoBeta Fenix Beta uses GeckoView Beta. **fenixProduction** Fenix Production uses GeckoView Beta (69) currently. * gradle.py/variant.py: Replace "abi" with "engine". * Disable enableUnitTestBinaryResources until we can switch to Android Gradle plugin 3.5. * Fenix nightly should use both geckoview nightly and beta * Updates automation to use apk splitting and support different engine
2019-08-21 17:32:01 +02:00
return all_tasks
def raptor(builder, is_staging):
2019-07-19 16:28:40 +02:00
mozharness_task_id = fetch_mozharness_task_id()
2019-09-13 12:48:22 +02:00
gecko_revision = taskcluster.Queue({
'rootUrl': os.environ.get('TASKCLUSTER_PROXY_URL', 'https://taskcluster.net'),
}).task(mozharness_task_id)['payload']['env']['GECKO_HEAD_REV']
Configure either geckoview beta or nightly at compile-time (#4851) * Remove "abi" product flavor and introduce "engine" product flavor. This patch will allow us to build Fenix against GeckoView Nightly and GeckoView Beta by introducing a new flavor dimension: engine = [geckoNightly, geckoBeta]. In addition to that it adds a "fenix" prefix to the nightly, beta and production flavors to reduce the ambiguity between fenix beta/nightly and GeckoView beta/nightly. For now the build types have the following engine variants enabled: **debug**: geckoNightly, geckoBeta Both variants enabled for local development and testing. **forPerformanceTest**: geckoNightly, geckoBeta Both variants enabled unless the perf team only cares about Nightly (tbd) **fenixNightlyLegacy**: geckoBeta Uses GeckoView Beta for now - the same version we ship production builds with (same behavior as before). This release type will eventualyl be decommissioned once we switch to a separate Nightly app on Google Play. **fenixNightly**: geckoBeta Uses GeckoView Beta for now - the same version we ship production builds with (same behavior as before). Changing this build to use GeckoView Nightly is currently being discussed. **fenixBeta**: geckoBeta Fenix Beta uses GeckoView Beta. **fenixProduction** Fenix Production uses GeckoView Beta (69) currently. * gradle.py/variant.py: Replace "abi" with "engine". * Disable enableUnitTestBinaryResources until we can switch to Android Gradle plugin 3.5. * Fenix nightly should use both geckoview nightly and beta * Updates automation to use apk splitting and support different engine
2019-08-21 17:32:01 +02:00
variant = get_variant('forPerformanceTest', 'geckoNightly')
2019-09-13 10:08:51 +02:00
build_task = builder.craft_assemble_raptor_task(variant)
signing_task = builder.craft_raptor_signing_task(build_task['label'], variant, is_staging)
2019-09-13 12:48:22 +02:00
tasks = [build_task, signing_task]
for abi in ('armeabi-v7a', 'arm64-v8a'):
variant_apk = variant.get_apk(abi)
all_raptor_craft_functions = [
builder.craft_raptor_tp6m_cold_task(for_suite=i)
Configure either geckoview beta or nightly at compile-time (#4851) * Remove "abi" product flavor and introduce "engine" product flavor. This patch will allow us to build Fenix against GeckoView Nightly and GeckoView Beta by introducing a new flavor dimension: engine = [geckoNightly, geckoBeta]. In addition to that it adds a "fenix" prefix to the nightly, beta and production flavors to reduce the ambiguity between fenix beta/nightly and GeckoView beta/nightly. For now the build types have the following engine variants enabled: **debug**: geckoNightly, geckoBeta Both variants enabled for local development and testing. **forPerformanceTest**: geckoNightly, geckoBeta Both variants enabled unless the perf team only cares about Nightly (tbd) **fenixNightlyLegacy**: geckoBeta Uses GeckoView Beta for now - the same version we ship production builds with (same behavior as before). This release type will eventualyl be decommissioned once we switch to a separate Nightly app on Google Play. **fenixNightly**: geckoBeta Uses GeckoView Beta for now - the same version we ship production builds with (same behavior as before). Changing this build to use GeckoView Nightly is currently being discussed. **fenixBeta**: geckoBeta Fenix Beta uses GeckoView Beta. **fenixProduction** Fenix Production uses GeckoView Beta (69) currently. * gradle.py/variant.py: Replace "abi" with "engine". * Disable enableUnitTestBinaryResources until we can switch to Android Gradle plugin 3.5. * Fenix nightly should use both geckoview nightly and beta * Updates automation to use apk splitting and support different engine
2019-08-21 17:32:01 +02:00
for i in range(1, 28)
] + [
builder.craft_raptor_youtube_playback_task,
Configure either geckoview beta or nightly at compile-time (#4851) * Remove "abi" product flavor and introduce "engine" product flavor. This patch will allow us to build Fenix against GeckoView Nightly and GeckoView Beta by introducing a new flavor dimension: engine = [geckoNightly, geckoBeta]. In addition to that it adds a "fenix" prefix to the nightly, beta and production flavors to reduce the ambiguity between fenix beta/nightly and GeckoView beta/nightly. For now the build types have the following engine variants enabled: **debug**: geckoNightly, geckoBeta Both variants enabled for local development and testing. **forPerformanceTest**: geckoNightly, geckoBeta Both variants enabled unless the perf team only cares about Nightly (tbd) **fenixNightlyLegacy**: geckoBeta Uses GeckoView Beta for now - the same version we ship production builds with (same behavior as before). This release type will eventualyl be decommissioned once we switch to a separate Nightly app on Google Play. **fenixNightly**: geckoBeta Uses GeckoView Beta for now - the same version we ship production builds with (same behavior as before). Changing this build to use GeckoView Nightly is currently being discussed. **fenixBeta**: geckoBeta Fenix Beta uses GeckoView Beta. **fenixProduction** Fenix Production uses GeckoView Beta (69) currently. * gradle.py/variant.py: Replace "abi" with "engine". * Disable enableUnitTestBinaryResources until we can switch to Android Gradle plugin 3.5. * Fenix nightly should use both geckoview nightly and beta * Updates automation to use apk splitting and support different engine
2019-08-21 17:32:01 +02:00
]
for craft_function in all_raptor_craft_functions:
2019-09-13 10:08:51 +02:00
raptor_task = craft_function(
2019-09-13 12:48:22 +02:00
signing_task['label'], mozharness_task_id, variant_apk, gecko_revision, is_staging
2019-09-13 10:08:51 +02:00
)
tasks.append(raptor_task)
2019-09-13 10:08:51 +02:00
return tasks
2019-03-27 15:23:03 +01:00
def release(builder, channel, engine, is_staging, version_name):
Configure either geckoview beta or nightly at compile-time (#4851) * Remove "abi" product flavor and introduce "engine" product flavor. This patch will allow us to build Fenix against GeckoView Nightly and GeckoView Beta by introducing a new flavor dimension: engine = [geckoNightly, geckoBeta]. In addition to that it adds a "fenix" prefix to the nightly, beta and production flavors to reduce the ambiguity between fenix beta/nightly and GeckoView beta/nightly. For now the build types have the following engine variants enabled: **debug**: geckoNightly, geckoBeta Both variants enabled for local development and testing. **forPerformanceTest**: geckoNightly, geckoBeta Both variants enabled unless the perf team only cares about Nightly (tbd) **fenixNightlyLegacy**: geckoBeta Uses GeckoView Beta for now - the same version we ship production builds with (same behavior as before). This release type will eventualyl be decommissioned once we switch to a separate Nightly app on Google Play. **fenixNightly**: geckoBeta Uses GeckoView Beta for now - the same version we ship production builds with (same behavior as before). Changing this build to use GeckoView Nightly is currently being discussed. **fenixBeta**: geckoBeta Fenix Beta uses GeckoView Beta. **fenixProduction** Fenix Production uses GeckoView Beta (69) currently. * gradle.py/variant.py: Replace "abi" with "engine". * Disable enableUnitTestBinaryResources until we can switch to Android Gradle plugin 3.5. * Fenix nightly should use both geckoview nightly and beta * Updates automation to use apk splitting and support different engine
2019-08-21 17:32:01 +02:00
variant = get_variant('fenix' + channel.capitalize(), engine)
taskcluster_apk_paths = variant.upstream_artifacts()
2019-09-13 10:08:51 +02:00
build_task = builder.craft_assemble_release_task(variant, channel, is_staging, version_name)
2019-09-13 10:08:51 +02:00
signing_task = builder.craft_release_signing_task(
build_task['label'],
Configure either geckoview beta or nightly at compile-time (#4851) * Remove "abi" product flavor and introduce "engine" product flavor. This patch will allow us to build Fenix against GeckoView Nightly and GeckoView Beta by introducing a new flavor dimension: engine = [geckoNightly, geckoBeta]. In addition to that it adds a "fenix" prefix to the nightly, beta and production flavors to reduce the ambiguity between fenix beta/nightly and GeckoView beta/nightly. For now the build types have the following engine variants enabled: **debug**: geckoNightly, geckoBeta Both variants enabled for local development and testing. **forPerformanceTest**: geckoNightly, geckoBeta Both variants enabled unless the perf team only cares about Nightly (tbd) **fenixNightlyLegacy**: geckoBeta Uses GeckoView Beta for now - the same version we ship production builds with (same behavior as before). This release type will eventualyl be decommissioned once we switch to a separate Nightly app on Google Play. **fenixNightly**: geckoBeta Uses GeckoView Beta for now - the same version we ship production builds with (same behavior as before). Changing this build to use GeckoView Nightly is currently being discussed. **fenixBeta**: geckoBeta Fenix Beta uses GeckoView Beta. **fenixProduction** Fenix Production uses GeckoView Beta (69) currently. * gradle.py/variant.py: Replace "abi" with "engine". * Disable enableUnitTestBinaryResources until we can switch to Android Gradle plugin 3.5. * Fenix nightly should use both geckoview nightly and beta * Updates automation to use apk splitting and support different engine
2019-08-21 17:32:01 +02:00
taskcluster_apk_paths,
channel=channel,
is_staging=is_staging,
)
2019-09-13 10:08:51 +02:00
push_task = builder.craft_push_task(
signing_task['label'],
Configure either geckoview beta or nightly at compile-time (#4851) * Remove "abi" product flavor and introduce "engine" product flavor. This patch will allow us to build Fenix against GeckoView Nightly and GeckoView Beta by introducing a new flavor dimension: engine = [geckoNightly, geckoBeta]. In addition to that it adds a "fenix" prefix to the nightly, beta and production flavors to reduce the ambiguity between fenix beta/nightly and GeckoView beta/nightly. For now the build types have the following engine variants enabled: **debug**: geckoNightly, geckoBeta Both variants enabled for local development and testing. **forPerformanceTest**: geckoNightly, geckoBeta Both variants enabled unless the perf team only cares about Nightly (tbd) **fenixNightlyLegacy**: geckoBeta Uses GeckoView Beta for now - the same version we ship production builds with (same behavior as before). This release type will eventualyl be decommissioned once we switch to a separate Nightly app on Google Play. **fenixNightly**: geckoBeta Uses GeckoView Beta for now - the same version we ship production builds with (same behavior as before). Changing this build to use GeckoView Nightly is currently being discussed. **fenixBeta**: geckoBeta Fenix Beta uses GeckoView Beta. **fenixProduction** Fenix Production uses GeckoView Beta (69) currently. * gradle.py/variant.py: Replace "abi" with "engine". * Disable enableUnitTestBinaryResources until we can switch to Android Gradle plugin 3.5. * Fenix nightly should use both geckoview nightly and beta * Updates automation to use apk splitting and support different engine
2019-08-21 17:32:01 +02:00
taskcluster_apk_paths,
channel=channel,
variant=variant,
# TODO until org.mozilla.fenix.nightly is made public, put it on the internally-testable track
override_google_play_track=None if channel != "nightly" else "internal",
is_staging=is_staging,
)
2019-09-13 10:08:51 +02:00
return [build_task, signing_task, push_task]
def release_as_fennec(builder, is_staging, version_name):
variant = get_variant('fennecProduction', 'geckoBeta')
channel = 'fennec-production'
build_task = builder.craft_assemble_release_task(variant, channel, is_staging, version_name)
signing_task = builder.craft_release_signing_task(
build_task['label'],
variant.upstream_artifacts(),
channel,
variant,
is_staging,
)
return [build_task, signing_task]
def nightly_to_production_app(builder, is_staging, version_name):
# Since the Fenix nightly was launched, we've pushed it to the production app "org.mozilla.fenix" on the
# "nightly" track. We're moving towards having each channel be published to its own app, but we need to
# keep updating this "backwards-compatible" nightly for a while yet
Configure either geckoview beta or nightly at compile-time (#4851) * Remove "abi" product flavor and introduce "engine" product flavor. This patch will allow us to build Fenix against GeckoView Nightly and GeckoView Beta by introducing a new flavor dimension: engine = [geckoNightly, geckoBeta]. In addition to that it adds a "fenix" prefix to the nightly, beta and production flavors to reduce the ambiguity between fenix beta/nightly and GeckoView beta/nightly. For now the build types have the following engine variants enabled: **debug**: geckoNightly, geckoBeta Both variants enabled for local development and testing. **forPerformanceTest**: geckoNightly, geckoBeta Both variants enabled unless the perf team only cares about Nightly (tbd) **fenixNightlyLegacy**: geckoBeta Uses GeckoView Beta for now - the same version we ship production builds with (same behavior as before). This release type will eventualyl be decommissioned once we switch to a separate Nightly app on Google Play. **fenixNightly**: geckoBeta Uses GeckoView Beta for now - the same version we ship production builds with (same behavior as before). Changing this build to use GeckoView Nightly is currently being discussed. **fenixBeta**: geckoBeta Fenix Beta uses GeckoView Beta. **fenixProduction** Fenix Production uses GeckoView Beta (69) currently. * gradle.py/variant.py: Replace "abi" with "engine". * Disable enableUnitTestBinaryResources until we can switch to Android Gradle plugin 3.5. * Fenix nightly should use both geckoview nightly and beta * Updates automation to use apk splitting and support different engine
2019-08-21 17:32:01 +02:00
variant = get_variant('fenixNightlyLegacy', 'geckoNightly')
taskcluster_apk_paths = variant.upstream_artifacts()
build_task = builder.craft_assemble_release_task(
variant, 'nightly-legacy', is_staging, version_name)
signing_task = builder.craft_release_signing_task(
build_task['label'],
Configure either geckoview beta or nightly at compile-time (#4851) * Remove "abi" product flavor and introduce "engine" product flavor. This patch will allow us to build Fenix against GeckoView Nightly and GeckoView Beta by introducing a new flavor dimension: engine = [geckoNightly, geckoBeta]. In addition to that it adds a "fenix" prefix to the nightly, beta and production flavors to reduce the ambiguity between fenix beta/nightly and GeckoView beta/nightly. For now the build types have the following engine variants enabled: **debug**: geckoNightly, geckoBeta Both variants enabled for local development and testing. **forPerformanceTest**: geckoNightly, geckoBeta Both variants enabled unless the perf team only cares about Nightly (tbd) **fenixNightlyLegacy**: geckoBeta Uses GeckoView Beta for now - the same version we ship production builds with (same behavior as before). This release type will eventualyl be decommissioned once we switch to a separate Nightly app on Google Play. **fenixNightly**: geckoBeta Uses GeckoView Beta for now - the same version we ship production builds with (same behavior as before). Changing this build to use GeckoView Nightly is currently being discussed. **fenixBeta**: geckoBeta Fenix Beta uses GeckoView Beta. **fenixProduction** Fenix Production uses GeckoView Beta (69) currently. * gradle.py/variant.py: Replace "abi" with "engine". * Disable enableUnitTestBinaryResources until we can switch to Android Gradle plugin 3.5. * Fenix nightly should use both geckoview nightly and beta * Updates automation to use apk splitting and support different engine
2019-08-21 17:32:01 +02:00
taskcluster_apk_paths,
channel='production', # Since we're publishing to the "production" app, we need to sign for production
is_staging=is_staging,
publish_to_index=False,
)
push_task = builder.craft_push_task(
signing_task['label'],
Configure either geckoview beta or nightly at compile-time (#4851) * Remove "abi" product flavor and introduce "engine" product flavor. This patch will allow us to build Fenix against GeckoView Nightly and GeckoView Beta by introducing a new flavor dimension: engine = [geckoNightly, geckoBeta]. In addition to that it adds a "fenix" prefix to the nightly, beta and production flavors to reduce the ambiguity between fenix beta/nightly and GeckoView beta/nightly. For now the build types have the following engine variants enabled: **debug**: geckoNightly, geckoBeta Both variants enabled for local development and testing. **forPerformanceTest**: geckoNightly, geckoBeta Both variants enabled unless the perf team only cares about Nightly (tbd) **fenixNightlyLegacy**: geckoBeta Uses GeckoView Beta for now - the same version we ship production builds with (same behavior as before). This release type will eventualyl be decommissioned once we switch to a separate Nightly app on Google Play. **fenixNightly**: geckoBeta Uses GeckoView Beta for now - the same version we ship production builds with (same behavior as before). Changing this build to use GeckoView Nightly is currently being discussed. **fenixBeta**: geckoBeta Fenix Beta uses GeckoView Beta. **fenixProduction** Fenix Production uses GeckoView Beta (69) currently. * gradle.py/variant.py: Replace "abi" with "engine". * Disable enableUnitTestBinaryResources until we can switch to Android Gradle plugin 3.5. * Fenix nightly should use both geckoview nightly and beta * Updates automation to use apk splitting and support different engine
2019-08-21 17:32:01 +02:00
taskcluster_apk_paths,
channel='production', # We're publishing to the "production" app on the "nightly" track
variant=variant,
override_google_play_track='nightly',
is_staging=is_staging,
)
tasks = [build_task, signing_task, push_task]
Configure either geckoview beta or nightly at compile-time (#4851) * Remove "abi" product flavor and introduce "engine" product flavor. This patch will allow us to build Fenix against GeckoView Nightly and GeckoView Beta by introducing a new flavor dimension: engine = [geckoNightly, geckoBeta]. In addition to that it adds a "fenix" prefix to the nightly, beta and production flavors to reduce the ambiguity between fenix beta/nightly and GeckoView beta/nightly. For now the build types have the following engine variants enabled: **debug**: geckoNightly, geckoBeta Both variants enabled for local development and testing. **forPerformanceTest**: geckoNightly, geckoBeta Both variants enabled unless the perf team only cares about Nightly (tbd) **fenixNightlyLegacy**: geckoBeta Uses GeckoView Beta for now - the same version we ship production builds with (same behavior as before). This release type will eventualyl be decommissioned once we switch to a separate Nightly app on Google Play. **fenixNightly**: geckoBeta Uses GeckoView Beta for now - the same version we ship production builds with (same behavior as before). Changing this build to use GeckoView Nightly is currently being discussed. **fenixBeta**: geckoBeta Fenix Beta uses GeckoView Beta. **fenixProduction** Fenix Production uses GeckoView Beta (69) currently. * gradle.py/variant.py: Replace "abi" with "engine". * Disable enableUnitTestBinaryResources until we can switch to Android Gradle plugin 3.5. * Fenix nightly should use both geckoview nightly and beta * Updates automation to use apk splitting and support different engine
2019-08-21 17:32:01 +02:00
if not is_staging:
tasks.append(builder.craft_upload_apk_nimbledroid_task(build_task['label']))
return tasks